estimate_effort_aerial_glmm() now reports a total
across the sampled days, the same basis estimate_effort()
uses, instead of a single average day (#363).
The two functions could be called on one design and returned quantities that differed by a factor of the number of sampled days, with nothing in either signature or either result to say so. On the package’s own aerial fixture that was 378.6 against 5092.5 — a gap a reader would naturally attribute to the diurnal correction the GLMM exists to apply. It is now 4728.6 against 5092.5, and the 7% between them is that correction.
target = "mean_day" restores the single-day figure for
anyone who wants it, and the basis is recorded on the result as
effort_target, so which question was answered can be read
off the object rather than inferred.
Both targets are expectations, and both now carry a retransformation
factor the previous code did not apply. The fitted curve is a
fixed-effects prediction — the day whose random intercept is zero —
which on a log link is the median day, not the mean. Reporting
it as a mean understates by exp(sigma^2 / 2): 4% on the
package fixture, more where days vary more. A mean_day
estimate is therefore about 4% higher than the old return value, not
identical to it.
The standard error scales with the expansion, so the CV is unchanged. It does not carry the uncertainty in the variance component itself, which makes it mildly optimistic; that is documented rather than hidden, and closing it needs a variance method neither the delta nor the bootstrap path offers.
This does not make the GLMM usable by the total estimators. They take no effort argument, and every design carries strata, so they always multiply per-stratum effort by per-stratum CPUE while this function returns one ungrouped row. Connecting the two needs stratified GLMM estimation, which is a modelling change and is tracked separately.
estimate_effort_aerial_glmm(boot = TRUE) now returns
NA confidence interval bounds when a multiplier’s
uncertainty was declared unknown, instead of an interval that quietly
left it out.
The total carries uncertainty from three places: the GLMM fit, the
visibility correction, and the angler-to-people ratio. Declaring one
unknown — visibility_correction = "none", or an
NA angler_ratio_se — makes the bootstrap skip
that component, because there is nothing to draw from. The standard
error already went NA for precisely that reason. The
interval did not: it was built independently from the bootstrap
quantiles and kept a value.
What that value described was an interval conditional on the unknown
multiplier being exact. Measured, with the same data and the same seed:
declaring the visibility correction unknown and declaring it known with
zero uncertainty produced bit-for-bit identical bounds of
[322.920, 449.732], distinguished only by se
being NA in one and 33.98 in the other. “Never
studied” was reported exactly as “studied, and found no uncertainty” —
the one equivalence this package must never assert.
The interval now goes NA alongside the standard error.
The delta path already behaved this way, because its interval is derived
from the SE and inherited the NA for free; the bootstrap
path built quantiles independently and had to say so explicitly. The two
paths now agree.
The point estimate is untouched — this is a reporting change, not an
estimation one. To get an interval back, state the correction’s
uncertainty (visibility_se, angler_ratio_se).
Declaring it known and zero is a claim you can make deliberately; it is
just no longer what silence means.
Examples no longer contain commented-out code. The
estimate_total_catch(),
estimate_total_harvest() and
estimate_harvest_rate() examples now run the lines they
previously only showed, and get_site_contributions(), which
had no runnable example, now has one. CRAN’s review of 7.0.0 asked for
this.
The validate_incomplete_trips() example and the
incomplete-trips vignette restore the equivalence-threshold option they
change.
Four vignettes described behaviour the package does not have. Each was found by running the vignette and reading its own output, not by reading the source.
incomplete-trips.Rmd told a reader whose validation had
just failed to “stick with
use_trips = "complete" (default)”. The default is
NULL, and on a roving design — which is what that vignette
is about — an unspecified use_trips selects the all-trip
mean-of-ratios estimator of Hoenig et al. (1997), not complete trips.
Following the parenthetical produced the estimator the line above it
warned against. Measured on the example data, catch rate differs by 30%
between the two (mean-of-ratios-cpue at 1.257 against
ratio-of-means-cpue at 0.967). The same claim appeared
twice more, once in a code comment. All three are corrected, and a new
section demonstrates the routing with a runnable example rather than
asserting it.
aerial-glmm.Rmd compared a one-day GLMM total against a
twelve-day total in a table headed “side-by-side comparison”, and
attributed the resulting 13.5x gap to the diurnal correction. The table
now states each row’s basis, and the like-for-like rows differ by 10.8%,
which is the diurnal correction the prose actually describes. Its
“Downstream Estimation” section also claimed that GLMM effort “feeds
directly into the standard downstream estimators”; it does not —
estimate_total_catch() takes no effort argument and derives
effort itself, so the section now says what the code does.
aerial-surveys.Rmd explained that ci_lower
/ ci_upper “give the 95% confidence interval” directly
beneath output where both were NA, because the design
declares visibility_correction = "none". Both aerial
vignettes now explain why an undeclared correction yields
NA rather than a number, and where to supply
visibility_se to get an interval.
camera-surveys.Rmd said negative-duration rows are “set
to NA and excluded from the daily sum”. They are excluded,
but the day reports a smaller number rather than NA — on
the package fixture one flipped ingress/egress pair takes a day from 11
hours to 7.75 — so the warning is the only place the exclusion is
visible.
The PDF reference manual builds again. Six characters across four
help pages had no LaTeX definition under pdflatex — tau,
kappa, pi, a combining macron, U+2212 minus and U+1D62 subscript i — and
stopped R CMD check from producing the manual at all. They
are now \eqn{} math.
The README names vignettes rather than linking to the published article pages.
Those vignettes ship inside the tarball, so
vignette("bus-route-surveys") works from an installed
package where a link to the website only works with a network. It also
means the README no longer depends on a checking machine being able to
reach the documentation host.
Every exported function now has a runnable example, and nothing
is wrapped in \dontrun{}.
Seventeen exported functions had no example at all, and fourteen help
pages wrapped theirs in \dontrun{}. Most of those wrappers
were not protecting anything: six ran as written once unwrapped, and the
rest referenced objects that were never defined — design_a,
my_effort, counts_df — so the block was a
sketch rather than an example. One demonstrated writing a CSV into the
user’s working directory.
Examples that genuinely need a suggested package are guarded with
@examplesIf rlang::is_installed(...) rather than hidden:
estimate_effort_aerial_glmm() needs lme4 and
summarize_by_county() needs zipcodeR. All 125 example
topics run, each in under five seconds.
DESCRIPTION cites the methods the estimators
implement — Hoenig, Jones, Pollock, Robson and Wade (1997) doi:10.2307/2533116 for
roving catch rates and Kinloch, McGlennon, Nicoll and Pike (1997) doi:10.1016/s0165-7836(97)00068-4 for bus-route designs
— and records the maintainer as copyright holder. The README shows
install.packages().
Printing a validate_incomplete_trips() result no
longer leaves the plot margins changed for the rest of the session.
print() widened par(mar) to draw its scatter
plot and never put it back; it now restores the caller’s setting on
exit.
simulate_creel_data() and
simulate_creel_catch() no longer leave the caller’s random
number stream reset.
Both called set.seed(seed) and never put back what was
there. seed is a convenience for reproducing one
simulation, not a licence to take over the session — but a script that
seeded its own analysis and then called either function part-way through
silently continued from our seed. Every draw after that point
was determined by an argument passed for one function’s benefit, with
nothing to say so.
Measured: after set.seed(999), the next
runif(1) was 0.389071 on its own and
0.685170 with a
simulate_creel_catch(seed = 42) call in between. Both now
give 0.389071.
The stream is captured before set.seed() and restored on
exit. A session that had never drawn at all is left without a
.Random.seed again, rather than inheriting ours.
Reproducibility is unchanged: the same seed still produces the same
data, and omitting seed still consumes randomness
normally.
Identifier columns are normalised to character at every join
(add_catch(), add_lengths(),
add_ages()).
A uid is a label, not a quantity: nothing is summed or ordered by magnitude on it, and what it must do is join. The CSV reader infers a bare integer id column as numeric while a JSON API serves the same ids as strings, so the same survey reached a design with a numeric uid from one backend and a character uid from the other.
Base R hid this — merge() and %in% coerce —
but dplyr::left_join() refuses outright with “Can’t
join x$interview_uid with y$interview_uid due
to incompatible types”, and identical() is silently
FALSE. Anyone joining a fetched interviews frame to a
fetched catch frame across backends hit it.
design$interviews, design$catch,
design$lengths and design$ages now carry
character ids whatever the source gave.
The coercion is deliberately not
as.character(): R renders a numeric 100000 as
"1e+05", so a naive conversion silently rewrites every id
at or above 1e5 — in the join key, the one column where a corrupted
value cannot be noticed by looking at a total. Whole numbers go through
sprintf("%.0f", ...), which is exact for every value R can
hold; a non-whole value is left alone rather than truncated into a
collision.
summarize_boat_composition() counts the events it
excludes (#337).
A count event yields an angler-boat share only when the boats were counted and some were present. Both exclusions are real; both happened with no trace that the event had occurred.
keep <- (ab + nb) > 0 is NA when
either count is NA, and an NA subscript does
not drop a row — it selects a phantom all-NA
one. The event survived the subset with an NA
month and day type, and aggregate(by = ) then dropped it
for having an NA grouping value. Two mechanisms chained,
neither visible: on a 12-day fixture, three unrecorded boat counts took
the event total to 9 and moved a reported share from
73.4% to 76.7%.
Two new integer columns, n_unknown_boats and
n_nonpositive_boats, close the accounting:
n_events + n_unknown_boats + n_nonpositive_boats
== count events in that month and day typeA month and day type whose every event was excluded now keeps its
row, reporting NA for pct_angler_boats rather
than disappearing.
An unrecorded grouping value is now genuinely pooled with a
category the data already records as "Unknown", rather than
forming a second group that merely prints under the same label
(#337).
#333 tracks missingness on an internal sentinel so a real
"Unknown" keeps its own counts. Where a column holds
both, that kept them separate through aggregation and then
rendered both as "Unknown" — so one recorded and one
unrecorded fish in the same length bin came out as two rows reading
N = 1 and 50% each, instead of one reading
N = 2 and 100%. The warning already emitted for this case
says the counts are pooled; now they are.
summarize_cws_rates() and
summarize_hws_rates() exclude interviews whose sought
species was not recorded, and report how many (#336).
The numerator counts fish of the species the party was targeting. With no target recorded nothing in the catch table can match, so such an interview fell through the join exactly as a party that caught none of its target does — and was scored the same way, as a zero. That asserted these parties caught none of something nobody recorded, and it dragged down every group they belonged to.
Measured on the shipped example data, blanking the sought species on
7 of 22 interviews: the boat group’s mean_rate went from
0.393 to 0.254, a 35% drop, with N
unchanged at 9. No row was dropped, no group was missing and
nothing warned — which made it harder to see than #333, where at least
something disappeared.
An interview whose effort was not recorded is
treated the same way, for the same reason: a rate needs an effort to
divide by. One unrecorded effort used to turn the whole group’s mean
into NA — mean() of anything containing an
NA is NA — while N went on
counting it. Those interviews are now excluded and counted in
n_unknown_effort, and the group keeps a real rate from the
rest.
Interviews whose effort is not positive are counted
too, in n_nonpositive_effort (#339). A zero is a real
record — a party interviewed before it started fishing — and a negative
one is a data error that add_interviews() already warns
about; neither yields a rate, and both used to be dropped with no trace
at all. A table could report 20 of 22 interviews with nothing in it to
say the other two existed.
Four changes to the returned table:
a new integer column
n_unknown_target;
a new integer column
n_unknown_effort;
a new integer column
n_nonpositive_effort;
N now counts the interviews that produced a
rate, not every interview in the group. The accounting closes
exactly:
N + n_unknown_target + n_unknown_effort + n_nonpositive_effort
== interviews in the groupThe three exclusion counts are mutually exclusive, in that precedence, so an interview missing more than one thing is counted once.
The estimand is now the rate among parties with a known target. That equals the rate among all parties only if the target went unrecorded independently of what was caught — an assumption about the data, not the code, which is why the count is reported beside every rate instead of the exclusion being silent.
A party that genuinely caught none of a recorded
target is a real zero and still counts, per add_catch(). A
group with no interview left to rate keeps its row, reporting
NA for mean_rate, se and the
interval.
The Calamus 2016 validation fixture ships again, so the tests that depend on it can run (#337 follow-up).
It lived under inst/extdata/, which
.Rbuildignore excludes — a correct exclusion when that
directory held 17 MB of real waterbody data and a pdfs/
directory of published journal articles. Those are long gone;
inst/extdata/ was down to the 32 KB fixture alone, so the
exclusion had stopped protecting anything and was only withholding the
one file it should have shipped.
The consequence was invisible. tidycreel.connect’s
test-composition-calamus.R resolves the fixture through
system.file(), so all eight of its tests skipped on every
run, in CI included — and they are that package’s only end-to-end
assertions against real reference numbers. That took connect from
437 passing with 9 skips to 461 with
1. (Stated as the delta rather than a running total: a current
count written into a release note is wrong by the next release.)
The fixture moved to inst/calamus-2016/ rather than
un-ignoring inst/extdata/, so the directory-level guard
stays in place: anything dropped in there later still cannot reach a
build.
summarize_length_freq(by = ) keeps every fish
(#337).
stats::aggregate(by = ) drops every row whose grouping
value is NA, so a length record with no recorded value for
a by column left the distribution entirely — taking its
weight with it. Measured on the shipped example data, blanking
species on 6 of 20 length rows took the total from
37 fish to 26: more than six, because a binned release
row carries a count rather than one fish.
The ungrouped total was never affected, which is what kept this invisible — and is why the existing tests could not fail on it.
An unrecorded grouping value is now reported under
"Unknown", sorted last, as it is in the six functions #333
swept. This is the last of that class: #333 could not reach this
function because its by selects columns of the lengths
frame, and no fixture built from the shipped data got that far.
summarize_cws_rates() and
summarize_hws_rates() no longer fail with
non-numeric argument to binary operator when every sought
species is unrecorded (#336).
Comparing anything with NA yields NA, and
an NA subscript selects a phantom all-NA row
rather than nothing (#324’s shape). Those phantoms made the filtered
frame look non-empty, so the aggregate returned zero rows with a logical
key and the join produced a non-numeric target count that failed later,
naming nothing the caller had set. The comparison now tests the sought
species for NA explicitly.
The two rate functions share one implementation of their grouping, rate and interval steps (#336). Those steps were byte-identical copies in both, which is how a seam fixed in one twin comes to survive in the other.
Six summary functions dropped an interview whose grouping value was not recorded, out of its own row and out of the total (#333).
table() and stats::aggregate(by = ) both
discard every record whose grouping value is NA.
summarize_by_angler_type(),
summarize_by_method(),
summarize_by_species_sought(),
summarize_successful_parties(),
summarize_cws_rates(by = ) and
summarize_hws_rates(by = ) all grouped that way, so an
interview with no recorded angler type, method or sought species left
the table entirely. sum(N) silently stopped equalling the
number of interviews attached to the design, and nothing errored or
warned.
Measured on the shipped example data, blanking 7 of 22 interviews:
every one of the six went from accounting for 22 interviews to 15, and
summarize_successful_parties() lost two whole rows. The
groups that survived lost their own members, so this was never only a
missing row — in summarize_cws_rates(by = "angler_type")
the boat group’s mean rate moved from 0.393 to 0.762,
because the interviews that vanished were the ones holding it down.
An unrecorded grouping value is now reported under
"Unknown", sorted last, matching
summarize_by_zip() and summarize_by_county(),
which have always done this. "Unknown" labels the absence;
it is never a category anyone selected, and nothing is imputed. (The
survey-weighted estimators use <unknown> via
group_value_labels(); the two conventions still
differ.)
This is the svyby() sweep of #321 reaching the functions
that sweep could not see: these group with base R, not
survey::svyby().
summarize_successful_parties() reports
NA, not 0, where success cannot be determined
(#333).
A party is successful when it caught some of the species it
sought. Where the sought species was not recorded there is
nothing to compare the catch against, so N_successful and
percent are now NA for those rows rather than
0 and 0.0%, which asserted that the parties
had failed. N_total still counts them: the interviews
happened.
An unrecorded angler type is a different case and is not blanked — the sought species is known, so whether it was caught is knowable, and only the reporting group is unknown. Those rows carry real counts.
summarize_cws_rates(by = ) and
summarize_hws_rates(by = ) report NA, not
0, for a group whose sought species was not recorded
(#333).
The numerator counts fish of the species the party was targeting.
With no target recorded there is nothing to count, and the upstream fill
made that count 0 — so the group reported a mean rate of
exactly 0.000, asserting that these parties caught none of
their target. Grouping by anything else leaves the rate determinable,
because the catch and effort are the interviews’ own, and those groups
are unaffected.
Missingness is tracked internally rather than inferred from the
"Unknown" label, so a category genuinely recorded as
"Unknown" keeps its own counts (#333). A sought species the
interviewer recorded as unknown is a real answer, not a missing one. A
column holding both unrecorded values and the literal value
"Unknown" now warns, because the two are pooled into one
row and cannot be told apart in the output.
summarize_successful_parties() no longer dies inside
base R when a grouping column is entirely unrecorded (#333).
aggregate() returned a zero-row frame and the failure
surfaced as replacement has 1 row, data has 0, naming
nothing the caller had set.
summarize_successful_parties() now reads
add_catch()’s catch-type model, so a party that recorded
only harvested/released rows counts as
successful (#329).
add_catch() documents the caught row as
optional: when a pair has none, its total catch is
harvested + released. This function held a private copy of
the rule with no such fallback — it asked whether a caught
row existed, full stop — so a party that recorded only its dispositions
was reported as unsuccessful while its own harvest was
positive.
This was live on the package’s own example data.
Four species-interview pairs in example_catch record
dispositions and no caught row — interviews 11, 12 and 14
for walleye, and 13 for bass. All four demonstrably caught the species
they sought, and all four were counted as failures. The successful party
total on the shipped data was 6; it is 10.
Dropping every caught row — which the documentation says
is legal, and which leaves each pair’s harvested and released rows
untouched — previously took the total from 6 to zero,
with every reported rate reading 0.0%. It now changes nothing, which is
the property that makes the rule real.
A recorded catch of zero still counts as unsuccessful: that is data,
not an absence. The fallback now comes from
species_counts_per_interview(), the package’s one
implementation of the catch-type rule, applied per species-interview
pair (#318, #320).
This was #317’s ninth instance and the last one outstanding; that defect class is now closed.
A large release, and a breaking one. The theme running through most of it is a single defect class: a quantity that was unknown or absent was allowed to behave like a zero or like nothing at all. Every instance produced a plausible number, no error and no warning.
Numbers that move, and why:
Grouped estimates gain rows.
survey::svyby() drops rows whose by= value is
NA, so any grouping column with a missing value silently
lost those records. estimate_total_catch(by = zone)
reported 565.2 fish against an ungrouped 784.6 — 28% of the
catch missing from a table that read as complete. Every grouped
estimator now reports the unknown group, so the parts sum back to the
whole (#317, #321).
Length-based standard errors get larger. The
bins’ covariance matrix was discarded, and
est_compliance(), est_mean_length() and
est_biomass() each rebuilt a variance assuming the bins
were independent. est_compliance() reported a standard
error 32% below an independently computed
survey::svyratio() reference; it now reproduces that
reference exactly. compliance_se,
mean_length_se and biomass_se all change.
Point estimates do not move (#311).
add_catch() refuses an unknown
count. count = NA was accepted and read as a
genuine zero everywhere downstream — the unknown and the zero produced
byte-identical output. Now an error (#324).
Length and age totals are on a catch basis. They previously expanded the measured subsample as if it were a census. Ratios were right; totals were not (#310).
If you pin any standard error from the length or age path, or read any grouped table, re-run before comparing to earlier output.
Grouped effort now reports an unknown group
instead of dropping it, and refuses a supplied within-day variance that
is NA (#317).
Two separate ways an unknown quantity was behaving like nothing at all.
The unknown group. survey::svyby()
drops rows whose by= value is NA, with no
warning and no row in the result. The effort did not move anywhere else
— it simply stopped being reported, so the grouped parts no longer
summed to the ungrouped whole. On a design whose gear was unrecorded on
three days, estimate_effort(by = gear) returned groups
totalling 662.5 angler-hours against an ungrouped total of 773:
110.5 hours, 14% of the fishery, gone silently. With
within-day variance attached the same request instead died with a bare
missing value where TRUE/FALSE needed, because the two
sides of the variance match built their group keys with different base
idioms.
NA is now a group like any other here, exactly as it
already is on the interview side. Its row appears with its own estimate,
standard error and n, and the parts sum back to the whole.
A group genuinely labelled "NA" remains a
different group — the two collided under
paste(), which renders a missing value as the string
"NA", and that collision also made
add_counts() report such a row as a repeated sampling
unit.
The supplied variance. add_counts()
accepts a precomputed within_day_var column and validated
nothing about it. The consumer reads a missing value as “this unit
had a single count, so its within-day term is zero” — true for a
unit absent from the table, false for one present with an unknown sum of
squares. Three unknown PSUs took se_within from 26.46 to
23.45 and the reported SE from 29.56 to 26.90, with no warning.
Supplying NA in within_day_var or
n_counts now raises
creel_error_na_within_day_var: an unknown variance
component is not a zero one, and this table cannot carry the
difference.
The party-size expansion component travels with it. That producer
keyed its groups with the old idiom, so an unknown group matched nothing
and the consumer read “no match” as “this group contributed no
expanded boats” — a confident zero in place of a real component.
Caught by the ensemble review: the unknown group’s
se_expansion was 0.0 where it is
0.8.
Grouped effort results gain a row wherever a
grouping column contains NA, and a counts table supplying
NA in either within-day column is now rejected at attach
time rather than silently understating the SE.
The sweep over the remaining grouped estimators followed in #321, below.
The length bins’ covariance now reaches the ratio consumers, which were reporting standard errors up to a third too small (#311).
svytotal() over the bin columns estimates a full
covariance matrix, and two_phase_rescale() propagates it.
It then died at the est_length_distribution() seam, and
est_compliance(), est_mean_length() and
est_biomass() each rebuilt a variance from the
diagonal alone — the quadratic form with every
off-diagonal set to zero.
The bins are a partition of the same fish, clustered within
interviews and rescaled onto a single reported total, so they are
strongly dependent. On the package’s own example data the maximum
off-diagonal correlation is 1.0. Against a
survey::svyratio() reference built independently in base
R:
| proportion | SE | |
|---|---|---|
est_compliance(), before |
0.5714286 | 0.1422436 |
svyratio() reference |
0.5714286 | 0.2093703 |
est_compliance(), now |
0.5714286 | 0.2093703 |
The point estimates agreed all along — only the uncertainty was wrong, which is why nothing looked wrong. The package standard error was 32% too small, on a proportion that gets compared against a legal size limit.
All three consumers now take the quadratic form against the full
matrix, and est_compliance() reproduces the independent
reference exactly. The matrix travels as an attribute keyed by reported
group, with bin labels as dimnames, so a consumer aligns to it
by bin rather than by position — a caller who keeps a
subset of rows gets that subset’s block rather than a misaligned
one.
When the matrix is unavailable — an object from an older version, or
one subsetted in a way that dropped the attribute — the independence
form is used and a creel_warning_bin_vcov_unavailable
warning says so. An absent covariance is unknown, not zero; falling back
in silence would restore the defect by the back door.
est_biomass() was the one consumer that documented the
omission. It is no longer an omission, and its @details now
says so, as do the other two.
Note for anyone pinning these numbers:
compliance_se, mean_length_se and
biomass_se all change, and in the direction of
being larger. The point estimates do not move.
add_catch() now refuses an unknown
count, which was indistinguishable from a genuine zero
everywhere downstream (#324).
add_catch() documents that an angler who caught none of
a species need not appear in the catch table at all. Every consumer
therefore reads a missing row as a catch of none. That reading
is right for an absent row and wrong for a row that is present but
carries NA, and nothing could tell the two apart: the
summaries left-join the catch table onto the interviews and fill the
join miss with zero, which swallowed the NA along with
it.
Changing a single harvested count on the package’s own example data:
| that pair’s count | summarize_hws_rates() |
se |
|---|---|---|
5 (known) |
0.2332251 | 0.07883204 |
NA (unknown) |
0.1682900 | 0.05512425 |
0 (genuine zero) |
0.1682900 | 0.05512425 |
The unknown and the zero produced byte-identical output — the rate understated and the confidence interval narrowed, with no error and no warning.
count may no longer contain NA (CATCH-07,
class creel_error_na_catch_count). Refused at entry rather
than repaired downstream, for the same reason #322 refuses an
NA within_day_var: the consumers’ reading of a
join miss is correct exactly when the table carries no unknowns, so this
is the one place that can make it true.
It also removes two accidents in the
caught >= harvested + released check (CATCH-04). An
unknown harvested count was zeroed by that check’s own
sub_total fill and so passed in silence. An unknown
caught count made the comparison evaluate to
NA, and indexing a data frame with NA yields a
phantom all-NA row — so it did abort, but for the
wrong reason, naming the offending pair as "NA/NA" and
telling the user nothing about which row to fix. Both are now
unreachable, and the refusal names the real pair.
The distinction this protects is pinned in both directions: an interview absent from the catch table still reads as a zero, exactly as documented.
Note that dropping a row states something rather than
nothing. A dropped "harvested" or "released"
row means none of that disposition; a dropped "caught" row
makes total catch derive from harvested + released instead.
The error message says which, so the remediation it suggests is not
itself a silent change of claim.
Every remaining grouped estimator now reports the unknown group too, and the ratio-estimation sample-size floor is applied to it (#321).
#317 fixed grouped effort. survey::svyby() drops rows
whose by= value is NA with no warning and no
row, and the other grouped estimators reached it through their own
paths, so each one went on dropping the group silently. The parts
stopped summing to the whole in three different places:
estimate_catch_rate(by = gear) and
estimate_harvest_rate(by = gear) reported those 32 and said
nothing at all about the other 16 — a third of the sample.estimate_total_catch(by = zone) on counts whose zone was
unrecorded on a third of the days returned 565.2 fish against an
ungrouped 784.6 — 28% of the catch missing, in a table that
reads as complete.proportion column, whose denominator always included the
unknown group, quietly summed to 0.64 — the one visible trace, in a
column nobody reads as a completeness check.Two seams behind those numbers were worse than the missing rows:
The sample-size floor never saw the unknown group.
n >= 10 per group was counted with
aggregate(.count ~ ., ...), whose formula method drops
NA rows — so a grouped ratio request was admitted or
refused on a denominator that had silently lost interviews. The floor
now counts on the group key, and an unknown group must clear the same
n >= 10 as any other.
A second private copy of the key rule.
expansion_stratum_key() built its own key with
paste(), which renders a missing value as the literal
string "NA" — so an unknown stratum and a stratum genuinely
labelled "NA" received the same key and had their
party-size expansion components pooled, with the total still looking
right. It now delegates to group_key(), the package’s one
answer to that question.
Grouped results gain a row wherever a grouping
column contains NA, in estimate_catch_rate(),
estimate_harvest_rate(),
estimate_total_catch(),
estimate_total_harvest() and the bus-route effort,
catch-total and catch-rate paths. A grouped ratio request whose unknown
group falls below n >= 10 now errors where it previously
returned an estimate for the other groups.
A reported grouping column keeps its own type.
Making the unknown group survive svyby() means promoting
the column to a factor — but only when it contains a missing value, so
the same estimator on the same column returned a factor when something
was unknown and a numeric when nothing was, and
depth > 15 or depth + 1 on the result
worked only in the second case. The factor is an implementation detail
of the svyby() call and is now undone on the way out: the
unknown group comes back as an NA of the column’s own type,
which is exactly what that column can say about a group whose value was
never recorded. This also repairs the grouped effort path shipped under
#317.
A column supplied as a factor comes back as that factor too, with its
own levels and its ordered class, and with a
true NA rather than the promoted
addNA() level — so is.na(result$group) finds
the unknown row for a factor exactly as it does for every other
type.
Undoing it exposed a defect the factor had been hiding. The
stratified product sum aggregated with
aggregate(cbind(...) ~ ., ...), whose formula method drops
NA rows — so an unknown group was dropped from the product
sum outright. It had survived only because an addNA() level
is not is.na() at the integer level, so the formula method
could not see it. That aggregation now splits on the group key, which
does not depend on the column’s type at all.
The sample-size floor’s message names the unknown
group. Its bullets were built with
paste0(by_vars, "=", vals), which renders a missing value
as the literal string "NA" — the very label a group may
genuinely carry. A design with both produced two bullets that both read
Group gear=NA: n=6, with no way to tell which group had
failed. An absent value now reads gear=<unknown>.
Two estimators named in the issue turned out not to
be reachable, and are recorded here so the question is not re-opened:
the camera path groups by strata rather than by a user variable, and
add_counts() rejects an NA stratum at Tier 1;
the aerial path calls svytotal() and supports no
by= at all. The sectioned paths group by a registered
section, and add_sections() rejects an unregistered
NA.
Species-level catch now reads add_catch()’s
catch-type model per species-interview pair, as that
model is documented, instead of once per species across the whole catch
table (#318).
add_catch() states the rule per pair: a
"caught" row is the pair’s total and is optional, and
when it is absent, total catch is inferred as
harvested + released. CATCH-04 enforces
caught >= harvested + released for each pair. The code
applied that rule table-wide instead — it asked whether a
"caught" row existed anywhere for the species, and
if one did, every other pair holding only
harvested/released rows read as a catch of
zero, while its own harvest stayed positive.
On the package’s own example data that made reported harvest exceed reported catch for all three species — walleye 33 caught against 55 harvested plus released, bass 10 against 25, panfish 7 against 13. No error, no warning, and the contradiction CATCH-04 rejects row by row was reintroduced at the total.
Species catch totals rise wherever a catch table
mixes the two shapes. This reaches
estimate_total_catch(by = species), the species CPUE path,
exploitation rate (which divides harvest by catch), and the bus-route
equivalents, all of which read the same helper. Tables recording
"caught" rows for every pair, or none at all, are
unaffected — the old rule and the new one agree there, which is why no
existing test moved.
An interview absent from the catch table entirely still counts as
zero for that species. add_catch() documents that too, and
the two absences are different: a missing row means the angler caught
none, while a missing "caught" row alongside recorded
dispositions is an instruction to derive.
This is the eighth instance of the pattern tracked in #317 — a quantity that is unknown or absent behaving like a zero.
est_length_distribution() and
est_age_distribution() now read the same per-pair rule when
they build the reported total they scale onto, and they refuse per
group rather than per species (#317).
Both distributions are rescaled onto a design-estimated reported
total (see #310 below). That total was built by a second, private copy
of the add_catch() catch-type model — the table-wide copy
#318 removed everywhere else. A species with a "caught" row
on any one interview therefore had every other interview’s
harvested/released rows read as a catch of zero, and the distribution
was scaled onto the understated total. On the package’s own example
data, grouping a catch distribution by species and an interview
attribute returned totals built from the "caught" rows
alone: 7 / 18 / 5 / 15 where the reported catch is 10 / 28 / 18 /
27.
The refusal moved with it. When no reported rows exist, these functions abort rather than scale onto zero — but the test asked whether the species had rows anywhere in the catch table. A group whose own interviews reported nothing still passed, because some other group’s interviews carried the species, and its measured fish were scaled onto a total of zero with no warning. The test is now whether any of the group’s own interviews records the species — on the key, not the value, so a recorded count of zero is still data.
Grouped length and age distribution totals rise
wherever a catch table mixes the two shapes, and a group that reports
nothing of its own now raises creel_error_no_rescale_total
instead of returning zeros. Shares (percent) are
unaffected. The refusal message now names the group, not only the
species.
This is #317’s own finding 7 — the same “absent behaves like a zero” pattern, in the rescaling path.
est_length_distribution() and
est_age_distribution() now scale their totals onto the
design-estimated reported catch instead of expanding
the measured subsample (#310).
Lengths and ages are read from a subsample of the catch. Expanding
that subsample through the interview design estimated the total
number of fish that happened to be measured — on the package’s
example data, 14 against a reported harvest of 77, with no error and no
warning. est_biomass() multiplies those counts by
weight-at-length and calls the result total biomass, so a headline
number was low by whatever fraction of the catch got measured. Measuring
every fish twice doubled the reported biomass.
The estimator is now two-phase (double sampling, Cochran 1977 §12.9 —
the structure already used for the camera calibration ratio): the bin
proportion among measured fish, scaled by the reported total. Both parts
come from one svytotal() call, so the bin-to-total
covariance is estimated rather than assumed away, and the standard error
is the delta method over it.
estimate, se and the confidence
bounds change value, as does every column of
est_biomass(). percent and
cumulative_percent do not: a share is invariant to how many
fish were measured, which is why the shape of the distribution was
always right and only its level was wrong.
est_mean_length() and est_compliance() keep
their point estimates for the same reason — both are
ratios. Their standard errors do move, because the per-bin standard
errors they read now carry the reported total’s variance through the
delta method rather than being a pure rescaling of the old ones: on the
example data mean_length_se goes 40.04 to 37.88 and
compliance_se 0.1554 to 0.1422. The cross-bin covariance
those two still assume away is a separate defect, tracked as #311.
Two consequences for existing code. Grouping by species now
requires add_catch(), because only that
table is species-resolved and scaling one species by the all-species
total would be worse than refusing. And a design with no matching total
— a harvest distribution with no harvest = supplied to
add_interviews() — is now an error rather than a subsample
total.
Every call that rescales warns, naming the measured total, the reported total and the factor between them.
targeted = FALSE on a by = species
request now tests that species’ own catch, not the total catch, under
the mean-of-ratios estimator (#304).
targeted = FALSE is documented as excluding zero-catch
trips so that the result is the rate among trips that caught the
species. The exclusion ran inside the mean-of-ratios branch, above the
species split, and tested the design’s total catch column. A trip that
caught one fish of any species therefore counted as non-zero however
many of the requested species it held, so on a survey where every
interview caught something the argument excluded nothing at all and
returned the untargeted rate with no warning:
# before: identical to the digit
estimate_catch_rate(design, by = species, estimator = "mor", targeted = TRUE)
estimate_catch_rate(design, by = species, estimator = "mor", targeted = FALSE)Both the exclusion and the accompanying “>70% of trips have zero catch” mis-specification warning now test the species’ own zero-filled count, which is the only reading that makes sense for a per-species rate. The regression estimator already did this (#290); the two now agree.
Three consequences for existing calls:
targeted = FALSE with by = species under
"mor"/"mortr" returns different numbers. It
previously returned the fishery-wide rate, so no caller loses a correct
estimate — but any recorded output changes.n >= 10 ratio floor, so a call that silently returned a
number now aborts. That is the intended trade: a targeted rate from a
handful of trips is not estimable, and an error naming the sample size
is preferable to an untargeted number wearing a targeted label.targeted = TRUE (the default) emits the
mis-specification warning per species where it previously could not
fire. No estimate produced with default arguments
changes.estimator = "ratio-of-means" continues to ignore
targeted, as documented.
Sectioned results now report their sections under the column the
design registered, instead of a hardcoded section
(#282).
add_sections(section_col = ) lets a design name its
section column whatever the survey calls it, but every sectioned
estimator returned a column literally named section
regardless. On a design registered with
section_col = reach:
# design registered with section_col = reach
names(estimate_catch_rate(design))
# before: "section" "estimate" "se" ...
# after: "reach" "estimate" "se" ...The result could not be joined back to the caller’s own section table
by name, and it disagreed with the package’s own error message:
by = reach is refused with “reach is already how the result
is split” while no output column was named reach. It was
also inconsistent with grouped results, which have always echoed the
caller’s column — by = day_type returns a
day_type column.
This affects all seven sectioned paths:
estimate_effort(), estimate_catch_rate(),
estimate_harvest_rate(),
estimate_release_rate(), and the three
estimate_total_*(). The lake-wide aggregate row is
unchanged in substance — .lake_total is a value in
the section column, so it now appears under the caller’s name too.
prop_of_lake_total and se_prop_of_lake_total
are not caller columns and keep their fixed names.
A design whose section column is already called
section is completely unaffected, which is every
fixture in this repo and the common case. That is also why no existing
test could catch this: with section_col == "section" the
hardcoded name and the correct name are the same string. The new tests
use a fixture that names the column something else, so they can
fail.
Code that reads est$section on a design registered under
a different name should read
est[[design$section_col]].
as_hybrid_svydesign() now takes one long-form
counts table plus a frame_col naming the
column that partitions it, instead of two pre-split tables named
access_data and roving_data (#248). The two
*_fraction arguments are replaced by a single
fraction, a named list with one entry per frame.
# before
as_hybrid_svydesign(
access_data = boat, roving_data = bank, calendar = cal,
access_fraction = c(weekday = 0.5), roving_fraction = c(weekday = 0.4),
trips_disjoint = TRUE
)
# after
as_hybrid_svydesign(
counts, frame_col = "angler_type", calendar = cal,
fraction = list(boat = c(weekday = 0.5), bank = c(weekday = 0.4)),
trips_disjoint = TRUE
)The arguments borrowed the interview vocabulary for
something that is not an interview mode. Access and roving describe how
anglers are interviewed and select the catch-rate estimator; tidycreel
carries that axis on add_interviews(interview_type =). What
this function combines are disjoint count frames,
usually angler-type domains. Pope et al. (Chapter 17) carry exactly this
as an anglerType column beside the stratum, which is what
frame_col now names, and taking a partitioning column
rather than pre-split tables is how the rest of the package already
works.
The frame labels now come from the data, so the design speaks the
caller’s vocabulary. attr(design, "component_col") names
the frame column, and the design data carries that column unchanged
rather than a renamed copy.
The two-frame ceiling is gone: three or more frames stratify, expand and weight exactly as two did. Nothing in the arithmetic was ever limited to two.
A frame label that is missing, and a frame_col holding
fewer than two distinct frames, are both refused rather than silently
forming a stratum. The repeated-day refusal is now keyed on the frame as
well as the date and stratum, so frames sampling a shared date are not
mistaken for repeat counts.
Two further refusals come with the caller-supplied labels. The
internal stratum key is paste(stratum, frame, sep = "."),
so a . inside either value can make two different
combinations land on one key – stratum "a" with frame
"b.c" and stratum "a.b" with frame
"c" both give "a.b.c" – which
survey would pool into one stratum, counting
n_h over the union of their sampled dates and getting the
day expansion and the fpc wrong for both. Ambiguous keys are now named
and refused; a . that cannot collide is still allowed. A
fraction entry naming a frame absent from the data, naming
one frame twice, or naming one stratum twice inside a frame’s vector, is
also refused rather than silently ignored, since each leaves the caller
believing a fraction was applied that never was.
The method on a sectioned catch rate now names the
estimator that produced it (#284).
estimate_catch_rate_sections() passed the caller’s
estimator down and computed with it, then labelled every result
"ratio-of-means-cpue-sections" whatever had run, so a
mean-of-ratios rate reported a ratio-of-means name – and that is the
roving default, because the auto-route resolves to "mor"
with no argument from the caller. A sectioned MOR catch rate is now
"mean-of-ratios-cpue-sections", or
"mean-of-ratios-truncated-cpue-sections" when
estimator = "mortr". Code matching on method
for sectioned catch rates should expect the new values. No estimate
changes; the numbers were already mean-of-ratios.
estimate_harvest_rate() and
estimate_release_rate() gained an estimator
argument and now resolve it from the design’s
interview_type, matching
estimate_catch_rate().
estimate_total_harvest() and
estimate_total_release() gained estimator and
truncate_at and follow their own rate functions through the
resolver added in #268 (#271).
Harvest and release rates and totals on a roving design
change by default. Where neither use_trips nor
estimator is specified and the design was built with
add_interviews(interview_type = "roving"), all four now use
all trips with the truncated mean-of-ratios estimator. Previously
mean-of-ratios HPUE and RPUE existed nowhere outside the bus-route path:
the standard internals were ratio-of-means only, so a roving survey got
a mean-of-ratios catch rate and a ratio-of-means harvest rate off one
design, with no message.
Hoenig et al. (1997) recommend the truncated mean of ratios for a roving survey because the clerk intercepts trips mid-stream. That argument is about the interview, not about which fish are counted — harvest and release are recorded in the same interception and are length-biased the same way.
To keep the previous numbers, pass
use_trips = "complete" or
estimator = "ratio-of-means"; naming either one suppresses
the routing. Access-point designs are unaffected. Bus-route and ice
designs never auto-route, and their totals now refuse a mean-of-ratios
estimator rather than accepting and ignoring it.
This completes #268. The rule established there is unchanged — each total resolves by the same rule its own rate function uses — and only what that rule resolves to has moved, because the harvest and release rate functions now have an estimator to follow.
The method on a harvest or release estimate now
names the estimator that produced it: "mean-of-ratios-hpue"
/ "mean-of-ratios-rpue" (and the -truncated-
and -sections variants) rather than always reporting
"ratio-of-means-". Code matching on method for
these two metrics should expect the new values on MOR paths.
estimate_total_catch() now resolves its rate
estimator from the design’s interview_type instead of
always requesting ratio-of-means, and gained estimator and
truncate_at arguments (#268).
Total catch on a roving design changes by default.
On a design built with
add_interviews(interview_type = "roving"), and where the
caller specified neither use_trips nor
estimator, the total is now built from all trips using the
truncated mean-of-ratios estimator – the same specification
estimate_catch_rate() already chose for that design.
Previously the rate function used all-trip MOR while the total used
complete-trip ratio-of-means on the same object, so a survey’s reported
catch rate and its reported total catch came from different estimators
and different trip sets, with no message.
Mean-of-ratios and ratio-of-means are not two spellings of one quantity: MOR averages per-interview ratios and is the estimator justified for mid-trip interception, and Hoenig et al. (1997) recommend it, truncated at 30 minutes, “to estimate catch rate and hence total catch under the roving creel survey design”. The truncation is part of the estimator rather than a tuning knob, because the untruncated mean-of-ratios estimator has infinite variance.
To keep the previous numbers, pass
use_trips = "complete" or
estimator = "ratio-of-means" explicitly; naming either one
suppresses the automatic routing. Access-point designs are unaffected,
as are bus-route and ice designs, which estimate a completed-trip
Horvitz-Thompson total, never auto-route, and now refuse a
mean-of-ratios estimator rather than accepting and ignoring it.
estimate_total_harvest() and
estimate_total_release() are unchanged:
estimate_harvest_rate() and
estimate_release_rate() offer no estimator selection to
follow, so routing their totals would have re-created the same
rate-versus-total disagreement in the other direction. All three totals
now resolve through one function, so when those two rate functions grow
estimator selection (#271) their totals follow without further
change.
use_trips on all three
estimate_total_*() functions now defaults to
NULL (“not specified”) rather than "complete",
so a design that should be routed can be told apart from a caller who
asked for complete trips. "complete" remains what an
unspecified value resolves to everywhere except the roving catch case
above. An unrecognised value is still refused, with the wording now
“Must be one of” rather than match.arg()’s “should be one
of”.
estimate_total_harvest() and
estimate_total_release() gain a use_trips
argument and now estimate from complete trips by
default, and estimate_total_catch() honours
use_trips on the paths that were silently discarding it
(#266). This changes the numbers all three return on any design
that records trip status and holds a mix of complete and incomplete
interviews.
A rate estimated from incomplete trips is length-biased: an interview taken mid-trip reports the catch so far against the effort so far, and the two do not scale together over the trip. Excluding incomplete trips is the documented default of the rate estimators for that reason, and a total is effort times a rate, so it inherits the bias while still returning a plausible number.
Three separate failures, one shape:
estimate_total_harvest() and
estimate_total_release() had no use_trips
argument and no trip filter on any path. They were
built from every interview, while estimate_harvest_rate()
and estimate_release_rate() default to the complete ones –
so the total and the rate on one design disagreed about which interviews
they came from.estimate_total_catch() threaded use_trips
into each dispatch branch separately. It reached the ungrouped and
grouped paths and was dropped at the call sites for
by = <species> and for sectioned designs, where the
argument was accepted, documented and inert: "complete" and
"all" returned the same number, which was the
"all" number.use_trips = "all", as estimate_total_catch()
already did. Those estimate a completed-trip Horvitz-Thompson total,
where an uncompleted trip would contribute catch-so-far under the
inclusion probability of a completed one.All three now apply one filter, once, before any dispatch, so the
ungrouped, grouped, species and sectioned paths are built from the same
interviews. The filter runs ahead of the
creel_warning_pooled_domain_mix check, so that warning now
describes the interviews the estimate is built from: under
use_trips = "complete" it previously reported a rate
spread, and per-level rates, drawn from the incomplete trips it had just
excluded. Passing the value down each branch is what allowed two of them
to lose it. The filter is deliberately quieter than the rate functions’
use_trips block: no messages, no minimum-sample abort, and
none of "incomplete", "diagnostic" or the
roving auto-routing, which belong to the rate a caller can estimate
directly.
Designs that record no trip status are unaffected – there is nothing
to filter on – and callers wanting the previous behaviour can ask for it
with use_trips = "all". The new argument sits after
target, matching estimate_total_catch(), so
positional calls that reached aggregate_sections or later
without naming them will land one argument short.
The totals still never consult interview_type and are
hard-wired to ratio-of-means, so a roving design’s total does not follow
its rate function to mean-of-ratios. That is a separate defect, filed as
#268.
estimate_total_harvest() and
estimate_total_release() no longer ignore a design’s
sections when grouping by species (#255). This changes the
numbers those two calls return on a sectioned design.
Both resolved species before the section dispatch, so
estimate_total_harvest(sectioned_design, by = species)
returned a lake-wide species total: effort estimated by pooling the
sections rather than per section, reported with no section
column and nothing to say the sectioning had been ignored. It was a
believable number for a different estimand.
estimate_total_catch() was not affected in the same way –
it dispatched to sections first and then failed, with tidyselect’s
“Column species doesn’t exist”, because species lives in
the catch table and so is in neither the counts nor the interviews.
All three now dispatch to the section path first and resolve species inside it, returning one row per section per species. Catch is apportioned against each section’s own whole effort, which is what makes a species total formable per section at all: it rides on whole effort rather than splitting it.
As with any other grouping on a sectioned design, a species result
carries no .lake_total row and no
prop_of_lake_total. Grouping decides that, not the absence
of by – a species-only call leaves no grouping columns
behind, and a gate keyed on those alone would have summed a lake total
across species.
A grouping variable alongside species must still be present in the
counts, since effort is genuinely split by it. That refusal is the same
creel_error_count_unobservable_by raised on unsectioned
designs (#241), rather than a second wording.
as_hybrid_svydesign() now requires a
calendar and estimates a period total
rather than a sampled-day total (#246).
access_fraction and roving_fraction are
within-day quantities – the proportion of a component’s frame the count
enumerated on a sampled day. Since #229 the PSU is the date, and those
fractions were passed straight to svydesign(fpc = ), which
is a stage-1 correction over the date PSUs. survey
therefore computed each stratum’s population as
sampled dates / fraction and read the result as a count of
days: three sampled dates at 0.5 made a “six day”
weekday stratum, where a June weekday stratum holds about twenty. A
fraction that should not shrink the stage-1 variance at all was
shrinking it as though half the calendar had been enumerated. Because
each component divided by its own fraction, access and roving also
implied different calendars for the same stratum – 6 days and 7.5 days
at once, the same arithmetic signature #229 fixed, displaced from within
a stratum to across the pair of them. The point estimate carried no
signal either way; only the standard error moved.
The population now comes from a required calendar
argument giving the days each stratum holds, counted as distinct dates
the way creel_design() counts them, and shared by both
components – one stratum is one span of the season, whichever method
observed it. Both expansions live in the weight: the within-day fraction
to the whole of a sampled day, and N_h / n_h to the season.
Only the second drives the finite-population correction.
Totals from this design are now season totals and are larger than
before by the day expansion. calendar has no default, and
every sampled date must appear in it under the same stratum.
as_hybrid_svydesign() now refuses a
calendar that assigns one date to more than one stratum
(#246).
A day counted in two strata lengthens the season by a day in each, so the period total expands to a calendar larger than the one that exists. On the probe fixture the weekend stratum grew from 5 days to 6 and the total moved from 1078.75 to 1195.5, with no error and no warning. A date repeated within one stratum is still accepted: counts distinct dates, so it changes nothing.
as_hybrid_svydesign() now refuses an
fpc that is not a non-missing logical scalar (#246). It is
branched on with a bare if, where NA surfaced
as base R’s missing value where TRUE/FALSE needed and a
length-2 vector silently took its first element – building the design
with a correction the caller never chose.
as_hybrid_svydesign() now requires Date
date columns and refuses a missing date or stratum in
access_data, roving_data or
calendar (#246).
Dates and strata are the keys the two components, the calendar and
the day expansion are all joined on, and they are compared through
as.character(), which renders NA as the string
"NA" and then matches it to every other NA. A
missing calendar date was counted as one more day in : on a four-day
weekday calendar with two sampled days, one NA row moved
the total from 156 to 195, with no error and no warning. A missing
calendar stratum silently withheld that day from the stratum it belonged
to, and a missing sampled date reached survey::svydesign(),
which aborted with missing values in `id' – an error naming
an internal column the caller never supplied. Date is now
required for the same reason the rest of the package requires it: the
keys must mean the same day on both sides of the join.
as_hybrid_svydesign() now refuses repeated counts on
one date (#246).
A day-level expansion is only defined when a sampled day is one row
per component. Two counts on a date are two looks at that date, not two
sampled days; they were summed, which multiplied the total by the number
of counts per day, and the new day expansion would have multiplied that
again. This is the defect class refuse_duplicate_psus()
(#193, #197) guards on the creel_design path. Clustering on
the date (#229) corrected the variance but left the point estimate
inflated, so the construction now aborts with a
creel_error_repeated_psus condition. Average repeats to one
row per date before constructing the design.
as_hybrid_svydesign() now requires
trips_disjoint, stratifies on the stratum-by-component
interaction, and clusters on the date (#229). Three seams, one
construction.
The two components were pooled into a single stratum while each kept
its own sampling fraction, so survey derived a population
size from a row count that mixed access and roving rows – one stratum
carried two population sizes at once (weekday was simultaneously 10 and
12.5 in the function’s own example), and the only signal was an
unexplained fpc' varies within strata warning from
survey next to a number that looked fine. Access and roving
sample different frames at different rates, so each is now its own
stratum and carries its own population size.
ids = ~1 made every row a PSU, so two
counts taken on one date were two independent sampling units – the
defect class refuse_duplicate_psus() (#193) exists to
prevent on the creel_design path, routed around by this
bridge. Observations are now clustered on date_col. Point
estimates are unchanged; standard errors were understated and are now
larger. A component that sampled only one date within a stratum now
leaves that stratum with a single PSU, so survey will
refuse to compute its variance where it previously returned one from
rows mistaken for days.
Adding the two component totals is valid only if the components
sample disjoint sets of angler trips, and nothing in date,
strata or count can establish that.
trips_disjoint is now required with no default: pass
TRUE to affirm the precondition holds.
component still names a survey method, not an
angler population – either method can cover either angler type – so
disjointness is a fact about the protocol and never about the
labels.
The documentation also said the returned object was “suitable for
effort estimation via estimate_effort()”, which refuses it:
it is a survey.design2, not a creel_design.
The help page now sends users to survey::svytotal(), and
its example is no longer wrapped in \dontrun{}, so
R CMD check executes the documented path. The text
described a hybrid design as suiting both components “within the same
sampling frame” – the condition under which the sum is wrong –
and gave differing coverage as an example of bias when differing
coverage is what makes the total valid.
The within-day variance component is now keyed by the sampling
unit rather than by the PSU alone (#227). add_counts() keys
design$within_day_var by the full unit key – the section,
the site, or whatever unit_cols named – and two consumers
rebuilt a narrower c(psu_col, strata_cols) key from the
design instead of reading the one the table was built with.
Nothing errored, because a join on too few columns does not fail: it
returns more rows than it was given. On a three-section design each
section’s 12 count rows matched three within-day rows apiece and became
36, so every section summed the lake-wide sum of squares, and the
inflated row count also became n_sampled in the variance
divisor. On a fixture where two of three sections are counted
identically at both count times – no within-day variation whatsoever –
all three reported the same se_within of 849.9, roughly 85%
of each section’s total standard error, against between-day components
of 110 to
228.
The same wrong key scaled the component to effort units.
ss_d is multiplied by T_d^2, and
match() on the date returned the first row carrying it, so
every section of a date was scaled by whichever section sorted first. A
section open 6 hours sitting alongside sections open 12 was scaled by
12^2 instead of 6^2: fourfold too large,
silently. Sections with different open hours is an ordinary field
situation.
This moves standard errors, confidence intervals and every
downstream product for any design whose unit key is wider than
(psu, strata): sectioned designs, site-structured designs,
and any use of unit_cols. Point estimates are unchanged.
estimate_total_catch(),
estimate_total_harvest() and
estimate_total_release() all build products from
estimate_effort_total() and inherit the correction.
The same defect crashed rather than lying when the extra unit-key
column came from unit_cols and the caller grouped by it:
the narrow join renamed the duplicated column and the estimator died
inside base R with replacement has 0 rows. That path now
returns per-group within-day variance.
Both consumers now read the key off the table itself, via the new
internal within_day_key_cols(), so a key written one way
and read another cannot recur.
Found by the sectioned/hybrid seam audit; the
.lake_total row’s standard error omitted this component
too, corrected separately below (#228).
The .lake_total row of a sectioned effort estimate
now reports the same variance components as the section rows above it
(#228). Section rows come from estimate_effort_total(),
whose se is sqrt(var_between + var_within).
The lake row came from a pure survey::svyby() +
svycontrast() aggregation, which carries the between-day
component and its across-section covariance and nothing else – two
definitions of variance in one column.
The result was a lake-wide standard error smaller than that of every section it contained. On a fixture whose three sections have genuine within-day variation the sections reported 405.6, 698.8 and 186.8 while the total reported 472.3, and that figure did not move at all when a section’s within-day spread was widened from 0.1 to 0.9.
se_between and se_within were reported as
NA on that row. The package’s convention is that
NA means unknown, so a missing component read as a
decomposition that could not be performed rather than one that was never
added; both are now reported.
The within-day component is second-stage sampling error inside one
unit, so on a shared day it is independent across sections and the
per-section components add; the between-day covariance the sections do
share is already inside the svycontrast() figure. A present
section carrying an unknown component propagates to an NA
lake se rather than to the between-day figure alone.
This widens the lake-wide standard error and confidence
interval for any sectioned design with more than one count per
day. Point estimates are unchanged, and a design with a single count per
day is unaffected: its within-day component is a true zero. Applies to
both method = "correlated" and
method = "independent".
Found by the sectioned/hybrid seam audit. Depended on #227: the correct per-section components are its input.
A sectioned effort estimate now carries the party-size expansion
component into its .lake_total row and reports it on the
returned object (#230). estimate_effort_sections() called
new_creel_estimates() without se_expansion,
and built the lake row’s variance from the between-day and within-day
components alone.
Both halves were wrong. se_expansion came back
NULL, which is this package’s signal that a component does
not apply and was indistinguishable from a design where no party-size
standard error was ever supplied. More seriously, the lake row’s
se did not depend on party_size_se at all: on
a two-section fixture the section rows moved from 15.811 to 15.937 and
from 7.906 to 8.108 when a party-size SE was introduced, while the lake
row stayed bit-identical at 14.577. It now reports 15.065.
The combination now routes through
combine_section_variances(), the helper the three
estimate_total_*_sections() twins already share, so the
structure classification cannot drift apart from theirs. A multiplier
estimated once and applied across sections is one random quantity common
to them, so its contributions add before squaring; per-section
multipliers are independent and combine in quadrature; groups that
straddle the sections unevenly are resolved exactly from the per-group
decomposition, and only fall back to NA when no
decomposition is available to resolve them.
That helper gained an expansion_in_section_var argument
to make this possible. The twins’ base is sum(section_var),
which already holds each section’s contribution as an independent term;
the effort base is between-day plus within-day, which does not, so the
independent terms are added before the covariance correction replaces
them. The argument defaults to the twins’ behaviour, leaving their
arithmetic bit-for-bit unchanged.
This widens the lake-wide standard error and confidence
interval for any sectioned effort design carrying a party-size
expansion with a supplied standard error. Point estimates are unchanged,
and a design with no expansion is unaffected – it still reports
se_expansion as NULL and an unchanged
se, rather than a zero that would be indistinguishable from
a component that never propagated.
expansion_decomposition is reported alongside the
component. The creel_estimates contract is one entry per
row of estimates, NULL exactly when
se_expansion is; returning a component with no
decomposition behind it broke that, leaving se_expansion
recoverable from nothing and giving a combination over a wider partition
no group index to work from. The .lake_total entry is keyed
by party-size group rather than by section – the groups summed across
the sections – so squaring and summing any row’s decomposition
reproduces that row’s component.
The group-wise sum is now
combine_section_decompositions(), factored out of
exact_expansion_var() so the aggregated variance and the
decomposition reported beside it cannot describe different
geometries.
Found by the sectioned/hybrid seam audit. The issue as filed described this as metadata-only; re-deriving it before implementing showed the lake row’s arithmetic was affected too.
Harvest and release rates on sectioned designs now honour
use_trips, and no longer include incomplete trips by
default (#263).
estimate_harvest_rate() and
estimate_release_rate() dispatched to their section path
before the block that validates use_trips, filters
the interviews and reports what it did. On a sectioned design the
argument was therefore inert: use_trips = "all" and
use_trips = "complete" returned the same number, an
unrecognised value was accepted rather than refused, and no filtering
message was emitted. estimate_catch_rate() has always
dispatched after that block, so on one design the three estimators
disagreed about which interviews they were built from – catch rate from
the complete trips, harvest and release from every interview.
This changes numbers. HPUE and RPUE estimated from incomplete trips are length-biased: an interview taken mid-trip reports the catch so far against the effort so far, and the two do not scale together across the trip. Excluding incomplete trips is the documented default for exactly that reason, and a sectioned design was silently opting out of it while returning a plausible-looking result.
Callers who want the previous behaviour can ask for it explicitly
with use_trips = "all", which now reaches the sectioned
path.
The sectioned path of estimate_total_catch() discards
use_trips for a different reason – the argument is never
passed to the section helper – and is tracked separately as
#266.
Grouped totals on sectioned designs now report the party-size expansion component too (#260).
A sectioned design splits by section; naming another variable in
by= splits within each section, and that branch delegates
to the grouped estimator – which reports a component per row of its own
result. Only the estimates were kept at the section boundary, so every
grouped sectioned total reported se_expansion = NULL while
its standard error carried the party-size term.
This is the same reporting gap as #259 one branch over, and it
outlived that fix in a way that made it visible: afterwards
by = c(<species>, day_type) reported the components
that by = day_type still returned NULL for, on
the same design and over the same partition.
Point estimates and standard errors are unchanged. An absent
section’s placeholder row reports NA, as it does for
species.
Species totals now report the party-size expansion component their standard error carries (#259).
estimate_total_catch(by = <species>) and its two
near-twins reported se_expansion = NULL on designs whose
counts were expanded from boats by an estimated party size, while the
standard error demonstrably moved with party_size_se. A
reader decomposing the standard error was told the party-size
contribution was absent when it was inside the number – the one thing
the component contract forbids, since NULL is how the
package says a term was never propagated.
The component was computed correctly and then dropped. The species
estimators move the species column to the front of the result, and on
the ungrouped product branch that result is a base data.frame, whose
[ keeps only names, row.names and class. The grouped branch
returns a tibble, which preserves attributes, so
by = c(<species>, x) was unaffected and the gap was
invisible there. On sectioned designs a second gap sat downstream: the
wrapper filled its per-section component only on the ungrouped branch,
so a sectioned species total reported nothing for its own reason.
Point estimates and standard errors are unchanged – the term was
always in se. What changes is that
se_expansion and the party_size entry of
se_components are now populated, one value per reported
row, with NA for the placeholder row a missing section
contributes.
Totals pooled over a domain the counts never classified now say so (#242).
Counts bound what a total can be broken down by. When a domain is
present in the interviews but not in the counts, the only available
total is E_total * rate_pooled, and the pooled rate is a
ratio of means weighted by the interview sample’s composition
over that domain. Had the domain been classified in the counts it would
be a stratum, and the total would be sum(E_h * rate_h) –
unbiased whatever the interview composition.
The two agree only when the interview sample’s effort composition matches the true effort composition, and interview selection is not proportional to effort by construction: access interviews intercept completed trips, over-representing anglers who must return to a fixed point, and roving interviews are length-biased toward longer trips. Malvestuto (1996) states it directly – “it is usually impossible to sample all angler types proportional to their level of effort”, a particular problem for bank anglers “widely dispersed along the shoreline”. So the mix differs by design rather than by accident, and where levels differ in rate the pooled total inherits that difference. Nothing in the reported output distinguished the two situations.
estimate_total_catch(),
estimate_total_harvest() and
estimate_total_release() now raise a warning of class
creel_warning_pooled_domain_mix when the interviews carry
an unclassified categorical domain and a crude rate
differs by more than 20% across its levels. Both conditions are
required: a domain the counts miss is only a problem when the levels
actually differ in rate, and a warning that fires where nothing is wrong
is one users learn to ignore. It fires once per estimator and domain,
and on the sectioned paths as well.
The screen is a ratio of sums taken straight off the interview
columns, not a survey-weighted estimate.
estimate_catch_rate(by = ) refuses sparse interview data,
which is precisely the case most at risk of a mismatched mix, so a check
built on it would fail where it is needed most.
The warning names a risk, not a defect, and is worded that way: this is unverifiable from within the data, because the counts hold no composition to compare against. The three estimators’ help pages gain a “What the pooled total assumes” section stating the same thing, and pointing at classifying the domain in the counts as what removes the assumption.
creel_schema() gains
harvest_lengths_table and
release_lengths_table, for a source that keeps harvest and
release lengths in separate tables (#185). Both fall back to
lengths_table, so a single-table source needs neither and
nothing changes for an existing schema.
tidycreel.connect’s YAML loader has offered both keys since #176 and passed them straight to this constructor, where they arrived as unused arguments: a profile setting either aborted with a base error naming no cause. The connect DBI backend reads the two lengths tables through them.
estimate_harvest_rate() and
estimate_release_rate() gain a targeted
argument, matching estimate_catch_rate() (#307).
targeted = FALSE restricts the domain to the interviews
that recorded some of the species being estimated, so the result is the
harvest or release rate among trips that took that species rather than
the fishery-wide rate. It is read by the mean-of-ratios forms and
ignored by ratio-of-means, exactly as for the catch rate,
and the per-species exclusion is now shared by all three rate functions
through one internal helper rather than copied.
targeted = FALSE requires
by = species on these two functions and raises
creel_error_targeted_needs_species otherwise. Without a
species there is no per-species count to test, and the only available
test would be “recorded nothing at all” — a different estimand, and one
the package has never estimated for harvest or release. Refusing is
deliberate: an argument that silently does nothing is the defect #304
fixed, and this avoids repeating it. The check runs above the section
dispatch, so it reaches sectioned designs too.
No estimate produced with default arguments changes. The per-species “>70% recorded none of this species” diagnostic warning is new on these two functions and fires only under the mean-of-ratios estimator.
estimator = "regression" now runs the regression on
a by = species request, instead of silently returning
ratio-of-means (#290).
The species dispatch sits above the regression route, so a species
request reached estimate_cpue_species(), whose only
branches were ratio-of-means and mean-of-ratios.
"regression" fell through to ratio-of-means. No error, no
warning, and a believable number under an estimator the caller did not
ask for. It affected flat designs as well as sectioned ones, which is
what separated it from #285.
reg <- estimate_catch_rate(design, by = species, estimator = "regression")
rom <- estimate_catch_rate(design, by = species, estimator = "ratio-of-means")
# before: identical estimates, method "ratio-of-means-cpue-species"
# after: method "regression-cpue-species", and the estimates differThe species form is CPUE₃ restricted to one species: the slope of
that species’ catch on the same angler effort, forced through the
origin, with the same leave-one-out jackknife SE. That is Petrere et
al. (2010) eq. 3, sum(C_i f_i) / sum(f_i^2), computed on
the trip set the call already uses. Both the flat and the sectioned
species paths route through it, so a sectioned request reports
"regression-cpue-sections" with per-section rows.
Zero-catch interviews are kept by default. An
interview that caught none of the target species contributes a
0 at positive effort, and it is a real observation: Petrere
et al. evaluated all three estimators with zeros present, using a delta
distribution with a 10% probability of zero precisely because “zero
catches are fairly common”. Dropping them would change the estimand from
the catch rate of that species per angler-hour to the rate among anglers
who caught it.
targeted = FALSE still makes that second choice
available, and on the regression species path only the
exclusion is applied per species: a trip that caught none of the species
being estimated is dropped, whichever other species it caught, and the
warning names the species and the percentage excluded.
It is confined to the regression form on purpose.
targeted has always been read inside the mean-of-ratios
branch, which tests the design’s total catch column and is
therefore usually inert on a species request — on the release fixture no
interview has zero total catch while 20 of 22 have zero bass. Widening
the per-species test to the other estimators would move numbers existing
ratio-of-means callers already get, so it is filed as #304 rather than
changed here.
This replaces the refusal added alongside #285, which was a deliberate placeholder while the modelling question was open.
Corrected the Petrere et al. (2010) reference in two more places (#290).
#233 fixed this citation in one block of
simulate_creel_data() and missed two others — a second
block in the same file, and the compare_cpue_estimators()
documentation. All three gave Fish. Res. 106: 325-333; the
paper is Braz. J. Biol. 70: 483-491, 10.1590/S1519-69842010005000010.
A grep for the journal name, rather than for the sentence, would have
caught all three at once.
@param targeted said zero-effort
trips were excluded; the code excludes zero-catch trips
(#290).
A species-level regression request was held to the
ratio-estimation n >= 10 floor (#290). That floor is a
ratio rule; the regression slope has its own “fewer than 3 interviews”
rule, and the ungrouped regression path was never held to the ratio one.
The species path therefore refused a defined estimator with a message
about a different one — reachable via use_trips = "all",
where the ungrouped regression ran at n = 5 while the
species form aborted. The floor still applies to the estimators it
belongs to.
A species-level regression result reported
variance_method as the caller’s variance
argument rather than "jackknife" (#290). The slope’s SE is
a leave-one-out jackknife computed inside the regression internals,
which never consult variance; the ungrouped and sectioned
regression paths already reported it correctly, so only the species path
named a variance that had not run — the same class of mislabel as
#284.
by = no longer accepts the interview id or an
internal .-prefixed column as a grouping variable
(#293).
On a design carrying catch data,
estimate_total_*(by = everything()) selected the species
column, routed to the species branch, and grouped by
every interview column — including
interview_id, which is unique per row. Nothing refused it.
On a 22-interview fixture the call did not return within 900
seconds.
The statistical stake is not the runtime. A key column as a grouping
variable puts one interview in each group, so every group’s rate has
n = 1 and no within-group variance is estimable. Had the
call returned, it would have produced a table of per-interview “totals”
each carrying an uncertainty that could not have been computed — the
failure mode this package treats as the dangerous one, since the number
looks fine.
Two refusals now sit in the shared by = resolver, so
every estimator that groups by interview columns gets them:
estimate_total_release(design, by = everything())
#> Error: `by` names the interview key `interview_id`.
#> x It holds one value per interview, so every group would be a single
#> interview and no within-group variance could be estimated.
estimate_total_release(design, by = .angler_effort)
#> Error: `by` names a column the package derived: `.angler_effort`.
#> x It is computed by `add_interviews()`, not data you supplied.Both tests are structural — they ask what the design registered, never what a name looks like or what values happen to hold.
The key comes from whichever of add_catch(),
add_lengths() or add_ages() registered the
interview id. Testing for distinct values instead would be wrong: on a
short survey a real grouping column such as date can be
unique per row without being a key.
The derived set is read from the design one field at a time, because
a leading . is not the test either. A user column literally
named .se_expansion is a supported grouping variable
(#259). And a design now records whether it computed
the trip duration, rather than the check inferring it from the column
being called .trip_duration_hrs: a caller may supply a
column of their own by that name, and it stays groupable. A column of
your own is never treated as derived, whatever it is called.
Derived columns are treated by how they were selected: a wildcard
such as everything() means every column the user brought,
so they are dropped silently, while asking for one specifically is an
error rather than a silent substitution. The two are told apart by
re-resolving the selector with the derived columns removed — if that
leaves nothing to select, the selector was asking for them. One
consequence is deliberate and worth knowing:
starts_with(".") is a refusal on a design whose only
dot-named column is derived, and a silent drop on one that also has a
user column such as .se_expansion.
estimate_effort() is unaffected: it resolves
by = against the counts, which carry neither an interview
key nor these derived columns.
Two limits worth stating. A design with no catch, lengths or ages
attached registers no id column, so an id there is still accepted —
nothing in the design says it is a key. And the sparse-group warning
still only warns at n < 3; whether n = 1
should be refused generally is a wider question this did not
settle.
creel_n_camera() no longer warns that a stratum is
below a Feltz and Middaugh (2025) camera-day minimum (#234). The 12
weekday and 7 weekend days that check used are the study’s
per-month well-performing schedule, while
n_h allocates over the whole period named in
N_h, so the comparison ran across scales and under-fired by
roughly the number of months surveyed. On the function’s own documented
example – about three months – the cited schedule is near 36 weekday and
21 weekend camera-days, but the check fired only below 12 and 7 and
passed a plan of 27 and 12 in silence.
Scaling the benchmark would have fixed only that one gap. Three
others remain and the function cannot close any of them: the 12/7
scenario is specifically at 1 count/day and nothing here knows the
counts per day; its error band is fixed by the study rather than taken
from cv_target, so a caller asking for a tight CV was
judged against the loose row; and the simulations measured boat-trailer
counts on six Arkansas reservoirs, whereas ybar_h and
s2_h are whatever the caller piloted. The benchmark is now
stated in ?creel_n_camera in the study’s own units, with
those conditions, as design context rather than a threshold.
Which benchmark applied was also chosen by matching
"weekday" or "weekend" as a substring of a
caller-supplied stratum name, so weekday_holiday took 12,
weekend_evening took 7 by luck, and Sat/Sun
took neither and drew a permanent “unclassified stratum” advisory
instead. No stratum name now changes the result.
No sample size changes. The check only ever emitted
a warning; n_h, total and
allocated were always returned as computed.
The error raised when a by variable is missing from
the interview data now carries the condition class
creel_error_by_missing_in_interviews and is attributed to
the estimate_total_*() call that produced it, rather than
to an internal helper (#254). Grouped totals require each
by variable in both the count and the interview data; the
count-side half already raised a classed, catchable condition and this
half raised an unclassed one, so only one of the two could be handled
programmatically. The message is unchanged.
estimator = "regression" now runs on a sectioned
design instead of being silently discarded (#285).
estimate_catch_rate() dispatches on sections before it
dispatches on the estimator, so a sectioned regression request fell
through to estimate_cpue_total(), whose estimator test
names only the mean-of-ratios variants – and "regression"
landed in the ratio-of-means branch. No error, no warning, and a
believable number.
The visible symptom was in compare_cpue_estimators(),
whose purpose is making estimator divergence visible: on every sectioned
design it reported the regression row as numerically identical to the
ratio-of-means row, carrying a jackknife standard error because that
function requests one for regression. A ratio-of-means point estimate
with a jackknife SE under the regression label corresponds
to no estimator in the literature.
A sectioned regression now fits one regression per section, on that
section’s interviews, and reports
method = "regression-cpue-sections" with
variance_method = "jackknife" – the variance that actually
ran, rather than the caller’s Taylor default. force_origin
reaches the sectioned path, which previously had no such argument at
all. The section-level jackknife SE rests on that section’s interviews
rather than the whole sample and is correspondingly less stable; this is
documented on estimate_catch_rate().
species in by combined with
estimator = "regression" is now refused rather than
answered with a different estimator (#290). The species dispatch also
sits above the regression route, so this affected flat designs
too, not only sectioned ones: the call returned ratio-of-means
numbers labelled "ratio-of-means-cpue-species" while the
caller had asked for regression.
Refused rather than implemented, because a per-species regression is a modelling decision and not a correction: a species that was not caught on a trip contributes a zero at positive effort, and whether those rows belong in the slope changes the estimate. That question is open in #290.
The mean-of-ratios diagnostic banner now describes the trips the estimate was actually built from, and every metric that takes the estimator prints one (#276). Two problems, both exposed by mean-of-ratios spreading beyond the catch rate.
HPUE got no banner at all. estimate_cpue_total() and
estimate_cpue_grouped() returned a mean-of-ratios object;
the harvest internals computed the same truncation metadata and then
discarded it, returning a plain result. The same
estimator = "mor" request therefore produced a caveat and a
truncation report for CPUE and RPUE and silence for HPUE. Harvest now
returns the same object its twins do, ungrouped and grouped.
The banner also said “This estimate uses incomplete trip interviews
(n of N total)” whatever trips had been used. That wording dates from
when mean-of-ratios was the incomplete-trip estimator; since
the roving auto-route (#268 for catch, #271 for harvest and release) the
default mean-of-ratios path uses all trips, so a roving
default rate announced an incomplete-trip caveat while using every trip
it had, and use_trips = "complete" announced one while
using none. The banner now names the trip set – “All Trips”, “Complete
Trips” or “DIAGNOSTIC: … (Incomplete Trips)” – and the length-of-stay
caveat and the validate_incomplete_trips() pointer appear
only for the incomplete set, which is what
mor_estimation_warning() already did at run time. The
truncation report appears on every path, because truncation is part of
the estimator rather than a diagnostic detail. The “n of N” denominator
is gone: trip filtering happens upstream, so N had become
the filtered count and the incomplete path printed a literal “24 of 24
total”.
The counts the banner reports are taken from the trips that survived truncation, not the set that entered it. Reported from before truncation they contradicted the truncation line printed directly beneath them – “over all 48 interviews” above “Truncation: 12 trips excluded”, when 36 ratios had been averaged. This affected all three metrics, in both the shared truncation helper and the catch rate’s own filtering block.
Interviews the rate internals discard are no longer counted as used.
Those internals drop missing effort, zero effort and missing catch or
harvest after the design-level counts are stamped, so a design with six
unusable interviews printed “over 48 interviews” beside an estimate
whose own n was 42, on all three metrics and on the grouped
paths.
Two things found while making the above change and fixed with it. A
mean-of-ratios rate reported no unit – the constructor had no
unit argument, so every MOR rate read NA while
the ratio-of-means rate beside it read "fish/angler-hour";
routing harvest through that constructor would have taken harvest’s unit
away. And the incomplete-trip count on a design with no trip status
column is now NA rather than 0, because that
count was never measured; the banner omits the clause instead of
reporting an absence as a zero. No estimate values change.
Estimates now record which estimator produced them, in a new
estimator component on the returned object (#275). A
total’s method names the product form –
"product-total-catch" whichever rate estimator built it –
so ratio-of-means, mean-of-ratios and truncated mean-of-ratios were the
same string, and the design slot carries the normalised
estimator, where a "mortr" request is indistinguishable
from "mor" at the default threshold. The field records the
estimator as you asked for it: "mortr" stays
"mortr". It is NULL on paths that take no
estimator argument, such as effort totals, which is deliberately
distinct from recording a default that was never chosen.
The sectioned rates gained the matching method labels
("mean-of-ratios-truncated-{cpue,hpue,rpue}-sections"), and
"mean-of-ratios-truncated-cpue" gained the display label
its HPUE and RPUE counterparts already had in print(),
format() and autoplot(). No estimate
changes.
The roving auto-route still resolves to "mor" rather
than "mortr" and still reports itself untruncated:
truncation runs at the 0.5 default there, but the caller did not ask for
it to be mandatory.
Naming the section column in by= on a sectioned
design is now refused on every rate estimator, instead of failing inside
tibble (#265). A sectioned result is already one row per section, so
estimate_catch_rate(d, by = section) asks for a split that
has happened; all three rate estimators answered with “Column name
section must not be duplicated”, raised by
tibble::add_column() and naming neither the design nor what
to do instead. The refusal and its wording already existed –
refuse_section_in_by(), error class
creel_error_section_in_by – and were wired into the three
totals only (#255). All three rate paths now use it, at the same point
in the call. Grouping within sections by anything else is unaffected,
and no estimate that returned a number before returns a different
one.
Two degenerate MOR truncation inputs now abort with tidycreel
wording rather than falling through to base R or to the survey package
(#279). truncate_at = NA passed the validator’s numeric and
length checks and reduced it to NA <= 0, so
if (NA) aborted with “missing value where TRUE/FALSE
needed”; all three validators – the catch-rate one, the totals
resolver’s, and the bus-route one – carried the same gap. A threshold
that truncates away every interview left an empty sample to reach
rowSums(), which aborts with “all arguments must have the
same length”, a message about matrix conformability for a condition
entirely about the chosen threshold; the refusal now names
truncate_at and the duration column, as the bus-route
incomplete-trip path already did. Both are pre-existing, and no estimate
that returned a number before returns a different one.
The MOR truncation message now reports a percentage of the
interviews it actually truncated. It divided by the incomplete-trip
count regardless of which trip set was being estimated, so
use_trips = "complete" with estimator = "mor"
divided by zero and reported Inf% – always taking the “high
truncation rate may indicate data quality issues” branch – and
use_trips = "all" reported the share of the incomplete
trips rather than of all of them, roughly doubling it on a
half-incomplete sample. Both the rate and the total paths were affected.
The denominator also excludes interviews with no recorded trip duration,
which are reported separately: a trip with no duration was never
eligible to be judged short, and counting it diluted the short-trip rate
enough to hide it below the 10% threshold that triggers the data-quality
warning. No estimate changes; the message is what a caller reads to
judge whether the threshold is discarding too much data.
estimate_catch_rate() no longer aborts when an
interview has no recorded trip duration and MOR truncation is in effect
(#272). The truncation filter compared duration against the threshold
without guarding for NA, and
NA >= truncate_at is NA: a logical index
carrying NA subsets a data frame to an all-NA
row rather than dropping it. That phantom row reached
svydesign() as a missing stratum and aborted inside the
survey package with missing values in 'strata', a message
naming neither trip duration nor tidycreel.
Trips with no recorded duration are now dropped, because a trip whose
duration is unknown cannot be shown to clear the threshold, and they are
counted and warned about separately from the short trips excluded by
truncation — a missing-duration loss is a data-quality fact, not an
estimator decision, and a caller needs to tell a threshold that excluded
six short trips from a duration column that is half empty.
mor_n_truncated now counts only short trips.
Affects every truncating path in estimate_catch_rate():
use_trips = "incomplete", use_trips = "all"
including the roving auto-route, and estimator = "mortr".
This is the guard truncate_interviews_for_mor() (#268)
already applied on the totals and
br_incomplete_harvest_rate() already applied on the
bus-route path; the standard rate path was the one site left without it.
Designs with a complete duration column are unaffected.
estimate_catch_rate() no longer aborts on a
bus-route or ice design built with
add_interviews(interview_type = "roving") (#270). The
roving auto-route fired before the bus-route/ice dispatch and was then
undone inside it, but the undo read a flag the route itself had to
clear, so use_trips reached the bus-route validator as
"all" — which it refuses. Such a design could not produce a
catch rate at all unless the caller passed use_trips
explicitly.
The route is now excluded at the point of resolution for these
designs, as resolve_total_rate_spec() already excluded it
and as the harvest and release rates get structurally by returning
first. Bus-route and ice results are unchanged, and a roving one now
matches the access-point one, as it should: these designs estimate a
completed-trip Horvitz-Thompson total, for which
use_trips = "all" names no estimator that exists. Standard
designs are unaffected.
estimate_harvest_rate() and
estimate_release_rate() can now group by species on a
sectioned design (#257).
Both failed with tidyselect’s Column \species` doesn’t
existwhileestimate_catch_rate()succeeded on the same design. On a sectioned design the public estimators return into the section path before their own species dispatch runs, and the harvest and release section helpers resolvedby=`
against the interviews alone – species lives in the catch table, so it
is in neither the interviews nor the counts.
Both now resolve by= the way the catch-rate path does
and estimate per species inside each section, from that section’s own
interviews. Grouping a further interview variable alongside species
works too, and a missing section still produces its placeholder row.
No count-observability constraint applies here, unlike the totals: a rate is estimated from interviews alone, so it may be grouped by attributes a total may not (#241). Nothing is apportioned and there is no lake row or share.
Grouping effort or a total by an attribute the counts do not
carry now explains the constraint instead of reporting the column as
non-existent (#241). estimate_effort(), the three
estimate_total_*() functions and their sectioned paths
resolve by= against the count data, which is correct –
effort comes from counts, so it can only be split by what the counter
could see. But the refusal arrived as tidyselect’s “Column
target doesn’t exist”, which is false to the user’s
situation: the column does exist, in the interviews, and
estimate_catch_rate(by = target) accepts it one call
earlier.
The message now says the column is in the interview data but not the
count data, lists the columns that can group effort, points at
estimate_catch_rate(by=) for a rate over that attribute
and, for the totals, at the by = <species> route that
apportions catch against whole effort. It also names the wrong
workaround the old message invited: copying the column into the counts
fabricates a classification the counter never made. Errors carry class
creel_error_count_unobservable_by; a column present in
neither table still raises tidyselect’s own error unchanged.
The three estimate_total_*_sections() product totals
now report se_prop_of_lake_total alongside
prop_of_lake_total (#243). The share of the lake-wide total
was the only quantity in the table without an error, so a reader
comparing sections had nothing to judge the comparison by.
This is not a port of the #231 fix for
estimate_effort_sections(). There the proportion is a
domain total over an overall total from one survey design, so
svyratio() returns the ratio and its error together. Here
prop_h = (E_h * rate_h) / sum_k (E_k * rate_k) has a
numerator and denominator that are each products of two estimates from
different designs – effort from the counts, rate from the
interviews – and the numerator is one of the denominator’s own terms.
The error is derived by delta method, carrying both the cross-section
covariance a shared party-size multiplier induces and the correlation
from the numerator appearing in the denominator.
The denominator variance and the cross terms come from the same
combine_section_variances() call that builds the
.lake_total row’s own standard error, so the reported error
belongs to the number beside it rather than to a parallel derivation
free to drift from it (#134). With two sections the shares sum to 1, so
one is a linear function of the other and their errors come out equal –
an identity a derivation that dropped the numerator-in-denominator
correlation would fail.
.lake_total reports 0, not NA:
its share of itself is exactly 1 by construction and was never
estimated. A section with no data reports NA for both
columns, since a zero would claim it held none of the total rather than
that nothing was observed there. Neither column is produced on the
grouped path, unchanged. No existing number moved.
The three sectioned product totals now report
expansion_decomposition alongside se_expansion
(#238). estimate_total_catch(),
estimate_total_harvest() and
estimate_total_release() each gathered the per-section
decomposition, used it in combine_section_variances(), and
then called new_creel_estimates() without it.
The constructor states the invariant where it stores the field: one
entry per row of estimates, NULL exactly when
se_expansion is. A component was reported with nothing
behind it, so se_expansion was recoverable from nothing and
a combination over a wider partition – a season, or several water bodies
– had no group index to work from. That index is what lets a “partial”
geometry be resolved exactly instead of refused, which is the mechanism
behind #150.
Each section’s entry is scaled by that section’s rate, as
se_expansion already was, so the per-row identity
sqrt(sum(decomposition^2)) == se_expansion holds on the
reported product scale. The .lake_total entry is keyed by
party-size group rather than by section – the groups summed across the
sections – because the section index is recoverable from the rows while
the group index is not.
No estimate or standard error moves. The lake row already consumed
the decomposition through combine_section_variances(); only
the reporting was missing. A design carrying no party-size expansion
still returns NULL for both fields.
Found while completing #230, which fixed the same defect in
estimate_effort_sections().
estimate_effort_sections() now forwards
target to the per-section estimator, and reports a standard
error for prop_of_lake_total (#231).
The target argument was dropped by a positional call, so
every section was computed on "sampled_days" while the
returned object was labelled with the caller’s target. The lake row read
design$survey directly rather than going through
get_effort_target_design(), so it was stuck on sampled days
too. On a calendar holding three times the sampled days,
target = "stratum_total" returned 104, 50, 154
– the sampled-day figures, unchanged in every row – while reporting
effort_target = "stratum_total". Nothing in the table
disagreed with anything else; only the label was wrong. It now returns
312, 150, 462.
This was unreachable from estimate_effort(), which
aborts for a sectioned design whenever
target != "sampled_days". That abort describes itself as
temporary, and this is the estimand-mislabelling class rather than an
arithmetic error, so it is cheaper to forward the argument now than to
remember it when the guard is lifted. Behaviour for
"sampled_days" is unchanged:
get_effort_target_design() returns
design$survey for that target.
prop_of_lake_total was a bare division of two survey
estimates, reported without uncertainty in a table where every other
quantity carries a standard error. It is a ratio of a domain total to
the overall total, both estimated from the same design and therefore
correlated – the denominator contains the numerator. The new
se_prop_of_lake_total column and the proportion now come
from one survey::svyratio() call, which handles that
correlation and keeps the reported error attached to the number beside
it rather than to a parallel derivation free to drift from it. The point
estimate is unchanged.
The .lake_total row reports
se_prop_of_lake_total = 0. That is a structural zero rather
than an unpropagated component: the lake total’s share of itself is
exactly 1 by construction and was never estimated, so NA
would assert an uncertainty that does not exist. An absent section
reports NA for both, since it has no share to report.
Found by the sectioned/hybrid seam audit.
estimate_total_catch(),
estimate_total_harvest() and
estimate_total_release() now document why they have no
targeted argument (#307).
A targeted rate is conditional on having recorded the species; total effort is not. Multiplying them applies a conditional rate to an unconditional base: on the package’s example data one species’ rate is 0.48 fish/hr over all 50 trips and 2.00 fish/hr over the 12 that caught it, so expanding the targeted rate by total effort returns roughly 223 fish where 30 were actually caught. The domain-consistent product needs the season-wide effort of species-catching trips, which no creel design observes. A targeted rate is therefore available and a targeted total is not, as a property of the estimand rather than a gap in the implementation.
impute_camera_counts() no longer attributes either
of its imputation models to a paper that does not contain it (#297).
Both citations named the wrong work. The negative binomial GLMM was credited to a real Afrifa-Yamoah et al. (2020) paper — but the group’s climate time-series paper, which imputes weather data with expectation maximisation and LSTM neural networks, and which the relevant paper itself cites for that purpose. The relevant one is
Afrifa-Yamoah, E., Taylor, S.M., Fisher, A. & Mueller, U. (2020). Imputation of missing data from time-lapse cameras used in recreational fishing surveys. ICES Journal of Marine Science 77(7-8): 2984-2994.
and swapping it in unqualified would have repeated the defect at a
finer grain: that paper evaluates nine models in a fully conditional
specification multiple-imputation framework and concludes that
zero-inflated Poisson models “were generally ranked
best”, reporting the negative binomial fits as slow and cumbersome to
converge. Its fixed effects are climatic covariates and its random
intercepts are temporal classes; neither appears here. It is now cited
for what it does support — the multiple-imputation framing behind
m > 1 and est_effort_camera_mi().
The Poisson GLM default was attributed in-text to “Hartill 2016”, with no matching reference entry to follow. Hartill et al. (2016) do impute camera outages with a GLM, but a cross-site one: the outage ramp’s daily count is predicted from the counts at two other ramps the same day, square-root transformed as third-order polynomials, given fishing year, season and day-type. The word “Poisson” does not appear in the paper, and their stated reason for a cross-site model is that same-ramp neighbouring days were “not considered to be sufficiently representative” — an argument away from, not towards, a local mean. Both papers now carry full reference entries saying what each does and does not support, and a new “Where these imputation models come from” section states plainly that the two models offered are the package’s own choices.
Also corrected in passing: the high-missingness warning told the user
that results “may be unreliable (Afrifa-Yamoah 2020)”, where that paper
reports “no clear systematic trend in the performance of the models with
respect to … the proportion of missing data” and successfully imputed
months of complete outage. The warning is kept — over half a stratum
being model predictions is worth saying — but it no longer claims a
source that says the opposite. A code comment describing the GLMM’s
(1 | site_col) term as a random slope now calls it a random
intercept, matching the formula and the site_col
documentation. The description also no longer calls
strata_col the model’s “sole predictor”: it partitions the
data, and a separate intercept-only model is fitted within each
level.
Documentation only; no estimate, imputed value or model changes.
Corrected two bad references in
simulate_creel_data() (#233) — one fabricated, one
mis-cited.
The reference read “Greene, B.T. (1995). The ANGLER simulation model. N. Am. J. Fish. Manage. 15: 743-750.” No paper by that title exists, and every field of the citation was wrong. The work the simulator actually draws on is
Greene, C.J., Hoenig, J.M., Barrowman, N.J. & Pollock, K.H. (1995). Programs to simulate catch rate estimation in a roving creel survey of anglers. DFO Atlantic Fisheries Research Document 95/99.
— a Department of Fisheries and Oceans technical report describing two S-PLUS functions that build an angler population and simulate a roving clerk, not a journal article, which is why no Crossref search for it returns anything. The author is Colin J. Greene with three coauthors, not a solo “B.T. Greene”, and it never appeared in North American Journal of Fisheries Management.
Found while verifying that one, and a different kind of error: the Petrere reference is a real paper, cited with the right title, authors and year, but given the wrong journal, volume and pages — “Fish. Res. 106: 325-333” for a paper published in Brazilian Journal of Biology 70: 483-491. Corrected, with its DOI added.
Su & Clapp (2013) was checked at the same time and is correct as cited.
The @details sentence claimed the generative model
“follows Su & Clapp
(2013) and Greene (1995)” without saying which part came from which. Greene et al.’s simulated anglers are deterministic — evenly spaced around the shoreline, all starting one hour into an eight-hour day, trip lengths alternating between 3 and 6 hours — so the three distributional levels are not from that paper. Only the roving-clerk step is: length-biased interception, with catch recorded up to the interview time. The docs now say so.
No computation changes. Documentation only.
Corrected the attribution of the camera calibration ratio, which
cited Hartill et al. (2020) for an estimator that paper does not contain
(#236). est_effort_camera() estimates rho, the
hours of effort per camera count, as a ratio of sums over the days
carrying both a count and interviews, and applies it to the stratum’s
full count total.
Hartill et al. (2020) is a review of digital camera monitoring. It presents no estimator and no variance, and where it discusses combining cameras with creel data it cites others. The earlier sweep in #235 replaced a fabricated Hartill reference with the real one and verified that the DOI resolved; it did not ask whether the resolved work supports the formula attached to it, which is a separate question that metadata cannot answer.
The citation is now split by what each source actually carries. The estimator and its variance are Cochran (1977): the counts are the first phase of a double sample and the interview days the second, which is the structure of Chapter 12 (Section 12.9, p. 343), and the ratio’s variance is eq. 2.46 with the finite-population correction omitted. The practice of calibrating camera counts against paired creel observations is credited to Hartill et al. (2016), van Poorten et al. (2015) and Eckelbecker et al. (2022) — each of which uses a different estimator: a per-day classification proportion, a hierarchical Bayesian model, and a fitted linear correction respectively.
?est_effort_camera now states plainly that the
ratio-of-totals form is this package’s own application of standard
double-sampling ratio estimation, not a reproduction of a published
fisheries estimator. In particular it is not Hartill et al.’s (2016)
rho, which is a dimensionless proportion of observed boats
that were fishing, estimated per day from interviews that are a
subsample of the camera’s own frame, with a bootstrap variance.
impute_camera_counts() also carried the Hartill et
al. (2020) reference, for a function that imputes camera outages by a
per-stratum Poisson GLM or a negative binomial GLMM. The review supports
neither, and the entry is removed.
No computation changes. This release alters documentation, comments and the camera vignette only.
Corrected the framing of as_hybrid_svydesign(),
which described access and roving as though they were
count methods (#246).
They are not. In the creel literature access and roving describe how
anglers are interviewed – access interviews intercept
completed trips as anglers leave, roving interviews intercept incomplete
trips while anglers are still fishing, and the two require different
catch-rate estimators (Pollock et al. 1994). A survey mixing them is a
hybrid interview design. Counts are described by their own
methods: instantaneous, progressive, bus-route, camera or aerial – the
values creel_schema() accepts for survey_type,
none of which is “access” or “roving”. tidycreel already carries the
interview axis on add_interviews() via
interview_type.
The help page previously stated that component “names a
survey method”, and the glossary defined a hybrid design as “fixed
access-point counts plus roving-route counts”. Both are now corrected:
the two components are two disjoint count frames, in
practice angler-type domains such as boat and bank anglers, and the
access/roving argument names are inherited
from the interview vocabulary and flagged as under review. No behaviour
changed.
Corrected five references that named papers which do not exist, or whose DOI resolved to an unrelated paper. Found by checking every DOI in the package against Crossref after the camera citation turned out to be wrong.
est_effort_camera(), estimate_effort_camera()
and impute_camera_counts(), gave a title, an author list
and a journal that belong to no paper, and a DOI
(10.1016/j.fishres.2020.105706) that resolves to a study of
age determination in sawsharks. The real reference is Hartill, Taylor,
Keller and Weltersbach 2020, Digital camera monitoring of
recreational fishing effort: applications and challenges, Fish and
Fisheries 21:204-215, .estimate_angler_n() and the mark-recapture vignette, used
10.1139/f58-002, which is The Abundance and
Distribution of the Northern Sea Lion. The correct DOI is
10.1139/f58-003; it is one article later in the same
issue.estimate_effort_aerial_glmm(),
example_aerial_glmm_counts and the aerial GLMM vignette,
had the right DOI but an invented title and the wrong pages, and the
vignette named four authors none of whom wrote it. It is Angler
effort estimates from instantaneous aerial counts, NAFM
38:194-209.simulate_creel_data(), is in Transactions of the American
Fisheries Society 142:234-246 under the title Evaluation of sample
design and estimation methods for Great Lakes angler surveys, not
in NAFM 33:895-909 under the title given.creel_n_camera(), was recorded as in press under a title
the paper does not carry. It is published as Improving efficiency of
estimating angler effort using low-frequency time-lapse camera
data, NAFM 45:322-332.No estimator changed. What changed is that following a reference now
reaches the work it claims to. Two related questions are tracked
separately: the provenance of the creel_n_camera()
camera-day minimums, which were attributed to the Feltz and Middaugh
title that does not exist (#234), and the unverified Greene 1995
citation in simulate_creel_data() (#233).
creel_n_effort() and creel_n_camera()
now share one internal implementation instead of holding two copies of
the same 37 lines (#295).
The two are the same stratified allocation reached through two
vocabularies — sampling days for angler contact, camera-days for a
camera deployment — and after #234 removed the camera-only warning their
bodies were byte-identical. Two copies of one computation is how a fix
lands in one twin and not the other, which this package has hit
repeatedly with the three near-twin
creel-estimates-total-*.R files.
No user-visible change. Both functions keep their exports, their separate help pages and their own vocabulary; validation moved into the shared internal but the checkmate assertions name the same arguments, so error messages are unchanged. Verified over 2,000 random inputs against the previous implementation: zero differences. A test now pins the two entry points as identical, so a future re-copy that edits one of them fails.
summarize_by_day_type() and
summarize_boat_composition() now resolve the day type
column instead of assuming it is the first stratum (#221). Both read
design$strata_cols[1] and labelled whatever they found
there day_type. But creel_design() preserves
the order the caller declared their strata in, so that index is a
declaration order, not a definition: a design declaring
strata = c(site, day_type) produced a table of site names
under a day_type header, with no warning, and the real
weekday / weekend breakdown absent entirely.
Resolution order is now an explicit day_type_col
argument, then a stratum actually named day_type, then the
first stratum – which warns and names the column it chose when the
design declares more than one. A single-stratum design resolves silently
and is unaffected, so the documented strata = day_type
workflow does not change.
This moves numbers for multi-stratum designs. On a
six-day two-site fixture whose boat composition is driven by site,
summarize_boat_composition() reported 90% / 10% – the
per-site means under a day_type header – where the
per-day-type figures are 63.3% / 36.7%.
summarize_by_day_type() moves labels rather than counts in
the balanced case, which is what made it invisible: the 6 / 6 site split
and the 6 / 6 weekday / weekend split are the same numbers.
A stratum has no canonical name in this package – the caller names
their own calendar columns – so this is a resolution with a documented
fallback, not a lookup. Pass day_type_col when neither
inference applies.
This is the same defect class as #216, which was the identical
strata_cols[1] shortcut on the camera calibration path.
Found while fixing that issue and recorded rather than fixed
inline.
estimate_effort() and the three total estimators now
refuse camera designs (#214). A camera count is a daily ingress total –
a count of arrivals – not an instantaneous count of anglers present. The
dispatch chain in estimate_effort() branches on
bus_route, ice and aerial, and
camera had no branch, so it fell through to the instantaneous path and
its counts were summed as though they were snapshots of how many anglers
were present. The result was a plausible number with a plausible
standard error: on the package’s own example data, 613 “angler visits”
where the calibrated estimator returns 111 angler-hours.
The camera vignette documented that route. It called
estimate_effort() for both sub-modes, stated that camera
designs “feed into the same estimate_effort() … pipeline –
no changes”, never mentioned est_effort_camera(), and
wrapped every call in suppressWarnings(). So the guards
added by #136, #137, #142 and #158 all sit in a function the
documentation never reached: a design carrying imputed counts, a stratum
with one paired interview day, a repeated count date, and an
uncalibrated raw expansion each went unreported on the documented
path.
The refusal is raised at all four entry points, not only in
estimate_effort(), because
estimate_total_catch(),
estimate_total_harvest() and
estimate_total_release() call
estimate_effort_total() directly and never pass through it.
Guarding only estimate_effort() would have left the totals
building a product from the same arrival count – multiplying a rate per
angler-hour by a count of arrivals and reporting it as fish.
Refusing rather than dispatching is deliberate.
est_effort_camera() already implements the calibrated
estimator and carries the guards; giving those guards a second caller to
be right about is how the split arose. It also takes arguments the
generic signature has nowhere to put – interviews,
n_anglers, h_open, calibration –
so a silent dispatch would have to guess them.
To fix an affected analysis, call
est_effort_camera(design, interviews = , n_anglers = ) for
the calibrated estimate, or
est_effort_camera(design, calibration = "none", h_open = )
to expand the raw counts under a declared assumption of one angler-hour
per count per hour open. Catch rates are unaffected – they come from the
interviews and never touch the camera – but there is no camera catch
total, and the vignette now says so rather than
demonstrating one.
estimate_angler_n(method = "schumacher", ci_method = "bootstrap")
is now refused rather than silently ignored (#209). The
Schumacher-Eschmeyer branch appended no
ci_lo_boot/ci_hi_boot columns and attached no
boot_samples, and raised nothing at all – so an explicitly
requested inference method vanished, and
estimate_mr_harvest(ci_method = "bootstrap") then aborted
telling the caller to do what they had already done.
The bootstrap is not implemented for this estimator on statistical
grounds rather than for want of effort. The other three methods resample
recaptures, m_k ~ Binomial(n_k, m_k/n_k), which is coherent
where the recaptures are the random component. Schumacher-Eschmeyer’s
published variance is the residual mean square of a weighted regression
through the origin (Seber 1982 eq. 4.17) – the scatter of the observed
points about the fitted line, which is a different quantity from
binomial noise in m. Resampling m_k alone
would report a narrower, differently-defined uncertainty under the same
column names.
This breaks any call that combined the two. Such a call previously
returned a correct point estimate and a correct regression interval, so
the fix is to drop ci_method = "bootstrap", which changes
nothing about the numbers returned. "logit" and
"delta" both give that interval. Use
method = "schnabel" where a bootstrap interval is genuinely
required.
Repeated sampling units with no count time are now refused at
estimation rather than warned about (#193). Two counts on one day are
two looks at that day, not two sampled days: the day’s effort is the
mean of its counts, and the spread between them is the within-day
variance component. That averaging has always been what
count_time_col triggers – but rows repeating a unit
without one bypassed it entirely and reached
svytotal(), which sums them. The reported effort came back
multiplied by the number of counts per unit (measured at exactly k-fold
for k = 1..4) and propagated undiminished into catch, harvest and
release totals, while se_within was reported as
0, indistinguishable from a within-day component that had
been evaluated and found to be nil.
add_counts() warned about this, and its sibling check
already aborted on rows identical in every column – so the
harmless case (a double entry) was refused while the dangerous one (a
genuine second count) was merely announced. The package cannot tell the
two apart from the table: rows sharing a unit key are either repeat
counts, which average, or undeclared distinct units, which sum, and only
the surveyor knows which. It now asks rather than guesses.
To fix an affected analysis, say what separates the rows –
count_time_col for repeat counts, or unit_cols
for distinct units – after which they are aggregated correctly and the
within-day spread is retained.
The refusal is raised by estimate_effort(), not
add_counts(), so estimators that never sum these rows are
unaffected: estimate_effort_aerial_glmm() models counts
against their flight time and keeps its several rows per day.
add_counts() still warns, so the problem is reported next
to the call that introduced it.
est_effort_camera() now estimates the calibration
ratio within every stratum the design declares, not within the first
stratum column only (#216). A design created with
strata = c(day_type, site) has strata day_type
x site, but the camera ratio path read
design$strata_cols[1] and keyed both the calibration and
the count total on it. One pooled hours-per-count ratio was formed over
the coarser partition and applied to counts belonging to a stratum that
never contributed to it.
Multi-column-stratified camera estimates change. On an eight-day
two-site fixture where north fishes 40 h on 10 counts and south 10 h on
100 counts, with interviews on three north days and one south day, the
estimate was 440 against a per-stratum truth of
200 – a 2.2x overestimate with no warning. The factor is
set by how unevenly interview effort is allocated across the dropped
columns, so it is unbounded in principle.
Where every day is an interview day the ratio of sums telescopes and
the point estimate is unchanged, but the standard error still moves: on
the balanced version of that fixture the calibration component was
107.2 against a within-stratum truth of 0,
because pooling two dissimilar site regimes inflates the ratio
residuals.
Estimating within the declared strata puts fewer paired days in each
stratum, so est_effort_camera() may now report an
NA standard error where it previously reported a number: a
stratum with one paired interview/count day has no measurable ratio
variance (#136), and a sum missing an unknown term is a lower bound
rather than a standard error. The warning names the stratum. Adding a
second matched interview day in that stratum recovers the SE.
interviews must now contain every column in
design$strata_cols. A missing one is an error naming the
column, where before the calibration proceeded on whichever columns the
table happened to carry.
Single-column-stratified camera designs – every fixture in the package’s own examples and tests – are bit-identical.
est_effort_camera() no longer treats a missing
camera count as a zero-effort day on the ratio-calibration path (#215).
na.rm = TRUE was passed to svytotal through
svyby, so an outage day’s count was dropped from the
numerator while its population day stayed in the frame – making it
contribute exactly zero hours to the total, with no error, no warning
and no message.
On the package’s own five-day fixture, setting one non-interview
day’s count to NA moved the estimate from
18.97 to 15.00, a 21% undercount. That
15.00 was bit-identical to the estimate obtained by
deleting the row outright, which is the demonstration: the missing day
contributed nothing while n still reported
5.
The raw-count branch of the same function passed no
na.rm and already returned NA for the same
input, so one function answered one input two opposite ways depending on
which branch it took. Both now return NA, and both now warn
– naming the affected dates, the camera_status values that
explain them, and impute_camera_counts() as the remedy. The
count is not imputed or reweighted here: which day is missing is
informative, so the treatment is the caller’s to choose.
Because missing rows are no longer dropped, survey::SE()
can now report NaN for a stratum whose total is
NA. That is normalised to NA_real_, since the
calibration component already uses NA for the same
condition (#136) and one function should not report one unknown two
ways.
The suppressWarnings() around the stratified count total
is removed, so survey’s own diagnostics reach the caller.
It previously swallowed every warning svyby raised, which
is half of why an outage produced a confident wrong number. It was also
unnecessary: the benign “No weights or probabilities supplied” note it
was presumably there for comes from svydesign() when the
design is built, not from svyby(), and removing the wrapper
surfaces no new warnings across the test suite.
Camera surveys with complete counts are unaffected.
The three total estimators now derive the reported
unit from their two factors instead of writing the literal
"fish" (#213). A total is "fish" only when a
per-angler-hour rate multiplies an effort in angler-hours; anything else
reports NA_character_.
There are two ways to fail to cancel, and both were labelled
"fish":
design$effort_unit is
NA whenever add_counts() received no
period_length_col, because a bare count column may be an
instantaneous head count or effort the caller already expanded, and
nothing can tell the two apart. Unknown times known is unknown.warn_party_hours_product() already reported that seam, but
the result still carried a confident label through it.This changes the reported unit for the common
workflow. The package’s own example_counts has no
period-length column, so a design built from it now reports
unit = NA on its totals where it previously reported
"fish". Point estimates, standard errors and confidence
intervals are unchanged – only the label moves. Supply
period_length_col to add_counts(), and
n_anglers to add_interviews(), to make the
unit derivable.
The same literal appeared three more times on the bus-route and ice
total paths (R/creel-estimates-bus-route.R), which reach a
different constructor. Those are keyed on
interview_effort_unit() rather than
design$effort_unit, since that is the effort those totals
are built from. Bus-route designs whose interviews carry a party size
are unaffected: their units already cancelled, and now they are shown
to.
This follows the rule estimate_effort_per_acre() already
used – compose the unit from its inputs, and an unknown input yields an
unknown result.
est_effort_camera() now reports the within-day
variance component instead of a literal 0 (#217).
add_counts(count_time_col = ) averages several counts on
one day into a daily mean and stores the within-day components
(ss_d, k_d) on the design; the camera
estimators never read them. The standard and aerial estimators have
always called compute_within_day_var_contribution() for
exactly this, so the machinery existed and only the call was
missing.
On a five-day fixture with two counts per day, widening the
within-day spread from zero to +/-30 counts – holding every daily mean,
and therefore the point estimate, fixed – moved
design$within_day_var$ss_d from 0 to
1800 per day and left the reported SE bit-identical at
3.266133. It is now 5.991888, with
se_within of 5.023454 where it was
0.
The component is scaled by the stratum’s calibration ratio on the
ratio path and by h_open on the raw path, because the
stored quantity is a variance of the stratum count total and each path
multiplies that total by a different factor. It is combined with the
between-day component at the variance level rather than by adding two
standard errors in quadrature, so a within-day variance of exactly zero
leaves existing estimates bit-identical.
se_within remains 0 for a design with one
count per day. That is the one case where a zero is right: there is no
within-day variation to measure, so the component is nil by construction
rather than unknown. Designs built without count_time_col
are therefore unaffected.
estimate_exploitation_rate() requires
se_C when C is a bare number (#208). It
previously reached if (se_C < 0) holding a
NULL and failed as argument is of length zero
– loud, so no wrong number ever escaped, but uninformative on an
entirely plausible call. The error now names the argument and points at
the object route added in #206, which supplies the standard error
itself. Defaulting the absent case to 0 was rejected: a
zero standard error cannot be told apart from a variance that never
propagated.
The reporting_rate documentation said the correction
adjusts the exploitation rate downward, contradicting the
formula printed beside it (#207). It adjusts upward:
under-reporting means the recoveries actually observed understate how
many tagged fish were removed, so dividing by lambda < 1
restores them and at lambda = 0.5 the estimate doubles. The
code was correct throughout and is unchanged; only the wording was
wrong. The direction is now also asserted in the test suite, so prose
and arithmetic cannot drift apart again silently.
estimate_exploitation_rate() now accepts the
estimate_total_harvest() result itself for C,
and checks it (#206). u = (C/T)(m/n)/lambda is the fraction
of the tagged cohort removed over the whole season, so
C must be a period total while T is the full
cohort – but estimate_total_harvest() defaults to
target = "sampled_days", and C arrived as a
bare number with its estimand stripped off. The shortest correct-looking
pipeline was therefore the wrong one, and it failed silently: on a
survey sampling 6 of 30 days the exploitation rate came back understated
five-fold, inside [0, 1] so the range guard never fired,
with a standard error that scaled down with it. The factor is the
sampling fraction, so sparser surveys were wrong by more.
Passing the object lets the target be read and a sampled-day total
refused. It also makes the catch-for-harvest substitution detectable –
released fish were never removed from the tagged cohort, and while both
totals are counts of fish, the method is recorded on the object. A
stratum total warns rather than aborting, since it is correct when
T is that stratum’s cohort. Supplying se_C
alongside an object is an error; the standard error is read from it.
Bare numeric C keeps working unchanged and now reports
that its target could not be verified. No estimate changes on any
existing call – the object and numeric paths return identical results
for the same total.
read_schedule() restored only four column types, so
window_id came back as character (#194). The column is
added by attach_count_times() rather than by
generate_schedule(), and
coerce_schedule_columns() matches an allow-list by name, so
a write_schedule() -> read_schedule() round
trip was not type-stable for it: a join against an integer
window_id, an arithmetic comparison, or an
identical() check silently saw a character vector.
window_id is now restored to integer using the same guard
period_id already used, so numeric ids become integers
while character window labels are preserved. No estimate changes –
schedules carry no quantities that reach an estimator.
The aerial-glmm vignette compared the GLMM against
the simple aerial estimator on one design holding four overflights per
day, with no count time declared. The simple estimator sums, so the
figure it published was four times the correct one (roughly 20,370
angler-hours against 5,092.5). The comparison now builds a second design
that declares the flights via count_time_col, aggregating
them to daily means; the GLMM continues to read the individual flights,
which is what it fits the diurnal curve against.
add_counts() now records unit_cols on
the design. It was previously validated and discarded, leaving the
design unable to distinguish a declared multi-column sampling unit from
an undeclared repeat.
estimate_effort() on an ice design renamed its
estimate column to record the effort type, so
tidy() returned total_effort_hr_on_ice (or
total_effort_hr_active) and no estimate at all
(#199). Ice was the only design to do this – including the degenerate
bus route, which is what an ice design is. Generic code reading the
documented accessor, tidy(x)$estimate, a rollup across
strata or species, or a report template, received NULL, and
sum(NULL) is 0: a season total came back as
zero rather than as an error. The effort-type column is now an alias
rather than a replacement, so both names are present and agree.
Bus-route and ice standard errors were computed over interview
rows rather than over the sampling unit (#198).
build_interview_survey() passed ids = ~1,
declaring every interview its own PSU. Malvestuto (1996, section 20.2.3)
defines this design as stratified two-stage probability sampling –
fishing days are the primary sampling units, and secondary units are
chosen within them – so several anglers contacted on one day are not
independent draws from the frame.
Reported precision was therefore a function of interview-recording
convention. Splitting one interview into two half-effort rows at the
same site left the Horvitz-Thompson estimate exactly unchanged and
shrank the standard error by 1/sqrt(2): an agency recording
one row per angler looked more precise than one recording one row per
party for the same survey.
The bus-route and ice estimators now use the ultimate-cluster
estimator, taking the variance between PSU totals, so partitioning
interview rows within a day leaves both the estimate and the standard
error unchanged. Access-point and roving designs are untouched – there
the interview genuinely is the sampling unit and ids = ~1
is correct.
No point estimate changes. Standard errors and
confidence intervals on bus-route and ice designs do change, and not all
in one direction: on the Calamus 2016 fixture catch_total
falls by more than half while effort_total roughly doubles.
Direction is a property of the data.
The stratified sample-size functions reported a
total that was not the sum of the per-stratum values, and
power_creel() rendered it as though it were (#195).
total is Cochran’s n, solved from the variance
equation before allocation; each stratum is then rounded up from it
independently, so the parts sum to as much as k - 1 more
for k strata. Printed beneath rows named after strata, in a
column named n_required, that row read as their sum and
under-booked the survey by the difference.
total is unchanged — it is a real quantity and was
documented as such. creel_n_effort(),
optimal_n() and creel_n_camera() now
additionally return allocated, the sum of the per-stratum
values, which is what the returned allocation commits to and the number
to budget against. power_creel() reports both rows. Code
reading these results by name is unaffected; code depending on the
length or exact names of the returned vector will see one more
element.
No estimate changes. This is a planning-stage reporting fix — the
per-stratum values were correct throughout, and rounding each up is
deliberate, keeping every stratum at or better than its share of
cv_target.
Expanded effort targets (target = "stratum_total"
and "period_total") understated the total whenever a
sampled day carried more than one count row (#183). The two sides of the
expansion factor N_h / n_h counted different things: the
numerator counted calendar rows, the denominator counted rows of the
attached counts table. A day holding k rows — two shift periods, three
spatial sections — therefore divided every weight by k, and the season
total came back low by exactly that factor, with no error and no
warning.
Both sides now count distinct sampling units, so the ratio is days over days however many rows a day carries. Rows sharing a day are summed into it before the expansion, which is what the Horvitz-Thompson estimator intends.
Two configurations change value, and both were wrong before: a design
whose counts carry a within-day dimension (the shipped
example_sections_counts expanded to 282 where the hand
calculation gives 846), and one whose calendar lists the frame at a
finer resolution than the day, which expanded as though the season held
more days than it does. A design with one count row per sampled day —
the shape of every existing test — is unaffected.
Estimates now report when a day carries several rows, so the reader can see that the quantity being expanded is a day rather than a row.
Registered sectioned designs never reached this:
add_sections() refuses expanded targets outright.
creel_schema() gains count_time_col,
naming the time a count was taken (#129). A count row is one observation
at one moment, not a day’s total, and sources routinely record several
on a sampled day; the time is the only thing that tells those rows
apart. Map it whenever the source records one and pass the fetched
count_time to add_counts()’s
count_time_col. Optional, and carried through as character
rather than parsed — it is a label that distinguishes observations, not
a quantity, and a source may write a clock time in any format.
creel_schema() gains length_bin_col and
length_count_col, the pair a source needs when it reports
released fish as length groups rather than measurements (#127). A binned
row is frequency-weighted — “350-400, 5 fish” is five fish — so the
count has to travel with the label. Both are optional and absent from
the required-column set: a source that measures every fish maps neither
and is unaffected. Map the label to length_bin_col rather
than length_mm_col, whose name asserts a unit the label
does not carry.
creel_schema() gains value_maps,
declaring what a source’s codes mean for the three columns whose meaning
is a fixed vocabulary rather than a number — trip_status,
catch_type, length_type (#128). Each entry
maps the source’s own codes to canonical values,
c("1" = "complete", "2" = "incomplete").
tidycreel.connect applies the map at the fetch, so a coded
source reaches add_interviews() speaking the vocabulary
every downstream filter matches. Map targets are checked against the
canonical vocabulary at construction, so a typo’d target is caught where
the map is written rather than several stages later against the
data.
New creel_vocabulary() returns those canonical
vocabularies. Exported because tidycreel.connect translates
source codes and must check its targets against the same list this
package filters on — a second copy would be free to drift from this
one.
creel_schema() gains strata_cols,
naming the stratum columns to carry through from the source (#171). It
is the one mapping here with no canonical tidycreel name on the other
side: add_counts() matches design$strata_cols
— the caller’s own calendar column names — against the names of the
counts frame, so the mapping is two-sided. Names are the column the
design refers to, values the source column holding it:
strata_cols = c(day_type = "DayType"). An unnamed entry,
c("day_type"), means the source already uses the design’s
name.
The advanced-use warning issued by
as_creel_svydesign() (formerly
as_survey_design()) no longer prints unevaluated cli
markup. It was raised with rlang::warn(), which does not
interpolate cli fields, so the line reached users as
Most users should use {.fn estimate_effort} instead. It now
uses cli::cli_warn(), as every other warning in the file
already did. The test covering the message could not have caught this:
its assertions sat behind an if (!is.null(result)) that was
never entered once the once-per-session warning had been consumed by an
earlier test, and it closed with expect_true(TRUE). It now
resets that state and asserts unconditionally.
The Calamus 2016 validation script now runs (#130).
inst/validation/calamus-2016-validation.R aborted at
add_counts() — the fixture carries three numeric count
columns and the call named none of them — so the package’s only
end-to-end validation of its own reference outputs had not executed at
all. It also called estimate_harvest_rate(), which returns
HPUE (0.4226 here), where reference-outputs.csv records the
Horvitz–Thompson total that estimate_total_harvest()
produces; a comment argued explicitly for the wrong one. Both fixed, and
the script now reports 3/3 estimands within tolerance.
tests/testthat/test-validation-guard.R can now fail
when that script is broken (#130). It previously accepted any error that
was not the working-directory guard — its comment said “any other error
(e.g. from load_all or estimators) is acceptable” — so it stayed green
for the entire period the script was aborting. It now asserts the script
runs to completion and that no estimand reports FAIL.
add_lengths() now accepts a length
column whose name is not literally length (#127).
length is one of this function’s own arguments, so an
unqualified length() call in its body made R force that
argument while searching for a function of that name, and any other
column aborted with object 'length_mm' not found before a
row was read. Every example and test passed
length = length, which resolves to
base::length and hid it — while the two names
tidycreel.connect’s fetch layer actually produces,
length_mm and length_bin, both failed. The
documented connect-to-design handoff for length data could not be run as
written.
Bus-route and ice totals now refuse a design with no complete
trips by name (#128). estimate_total_catch(),
estimate_total_harvest(),
estimate_total_release() and the bus-route rate estimators
filter to completed trips, and a Horvitz–Thompson assembly handed a
zero-row frame does not notice: it failed several calls later inside
rowSums() with
all arguments must have the same length, which names
nothing the caller can act on and reads like a package bug. The standard
designs already aborted by name here; these now say the same thing, name
the quantity that could not be produced, and point at
use_trips = "incomplete" or "diagnostic" —
never "all", which a bus-route design does not accept,
because an uncompleted trip supports a rate but never a total. No
estimate changes: every affected call already failed, just
unreadably.
print() on a creel_schema now groups
n_counted_col under interviews rather than
counts (#170). Both enumeration columns live on the interviews table —
add_interviews() resolves them against the interviews frame
and get_enumeration_counts() reads them back off it — so a
bus-route user reading the printed schema was told the enumeration count
belonged to a table it is not in, while its own denominator was listed
under another. Display only; no estimate was affected.
summarize_by_zip() and
summarize_by_county() gain a zip_col argument,
defaulting to "zip_code". Both previously required a
hardcoded raw field name from one agency’s database, which no
general-purpose package should assume. Rename the column, or pass
zip_col, to keep existing code working.
add_interviews() now warns rather
than informs when n_anglers is omitted (#126). The
assumption it states is a claim about the data, not a note about a
default: with any party larger than one, .angler_effort is
party-hours while count-derived effort is angler-hours, so every rate
denominator is wrong by the mean party size with no error raised. Pass
n_anglers = 1 to declare that the interviews really are one
angler each; that silences the warning and, unlike omission, marks the
effort as genuine angler-hours.
add_counts() refuses a counts table containing rows
identical in every column (#152). svytotal() sums the rows
of design$counts, so a repeated row was counted twice: a
six-day table rose from 65 to 77 angler-days and its standard error from
5.26 to 15.61, with only a warning. Previously CNT-06 warned; it now
aborts, naming the affected rows.
The check is on the whole row, not the sampling-unit key, and is
deliberately independent of unit_cols. Two rows sharing a
key are ordinary structure — two sections, two effort types, two counts
within a day — and differ somewhere. Two rows differing in
no column carry nothing that could distinguish one unit
from another, so the table is malformed under every key, including a key
that is wrong (as it has twice been: #155, #162).
Tables where the repeat is a genuine second observation are unaffected, since the counts themselves differ; CNT-06 still warns about those.
derive_angler_count() now removes the columns it
consumed (bank, boat_anglers,
boat_count) from its result. They are superseded by the
derived count and by expansion_basis, and leaving them in
produced a table that varied between sub-counts of one sampling unit —
indistinguishable, to add_counts(), from a structural
dimension it had not been told about (#162). The destination column is
never dropped, even when it is also an input. Code reading a raw
component off the result must read it from the input table
instead.
add_counts() aborts, rather than silently taking a
first value, when within-day aggregation would collapse rows that differ
in a column the sampling-unit key does not contain (#162). The error
names the column and supplies a ready-made unit_cols
call.
as_survey_design() is renamed to
as_creel_svydesign() (#167). The old name is srvyr’s
principal entry point, and srvyr is the natural companion for tidy
survey work, so attaching both packages masked one with the other
depending on load order. A user who loaded srvyr second and called
as_survey_design(design) got srvyr’s generic failing to
dispatch on creel_design, with an error that said nothing
about masking. The new name also matches the sibling
as_hybrid_svydesign() and states what the function does: it
extracts the internal survey object rather than
constructing a design. as_survey_design() keeps working and
now warns; it delegates to as_creel_svydesign(), so the two
cannot diverge.The calamus-2016 reference outputs record a
re-baselined catch_total standard error, 55.7239 becoming
52.9963 (#178). The point estimate is unchanged and always was. The file
was written once at v1.7.0 and never regenerated, so it had gone on
recording a number the package stopped producing at v3.0.0, when the
dimensional seam audit routed all three bus-route totals through
br_complete_trips_only() — a filter the harvest total had
always applied and the catch total never had.
Nothing about the estimator changed here; only the record of what it
produces. The fixture’s two incomplete-trip rows both carry
catch_count = 0, so dropping them cannot move a
Horvitz-Thompson sum — it moves only the interview count behind the
variance, 24 to 22. That is worth stating plainly, because the
divergence was first misread as evidence against the trip
filter on the grounds that the point estimate was invariant to it: with
zero-catch rows, invariance is guaranteed by construction and says
nothing about the SE.
The row was regenerated only after the responsible release was
identified from the source history and the pre-v3.0.0 value reproduced
exactly by disabling the filter on current code.
inst/extdata/calamus-2016/README.md is new and records that
reasoning, together with the standing rule that these outputs are not
re-baselined to match current behaviour without it.
effort_total and harvest_total are untouched
and have reproduced bit-for-bit since v1.7.0.
inst/validation/calamus-2016-validation.R now
compares standard errors as well as point estimates — six comparisons
where there were three (#178). Comparing estimates alone is the reason
the stale SE above survived three major versions: the script is the only
thing that exercises the reference outputs, and the one quantity that
had moved was the one it never looked at.
Its guard test gains two related fixes. It now asserts which
comparisons ran, not merely that none failed — a script that quietly
stopped checking standard errors would otherwise still report no
failure, which is the original blind spot one level up. And it no longer
wraps the script in suppressMessages(): the script reports
through message(), so suppressing them left the captured
output empty and the existing “no FAIL in output” check passing on a
zero-length vector, testing nothing.
The sampling unit is now declarable: add_counts()
gains unit_cols (#162). Until now the unit was inferred
from the design alone — the PSU column plus strata, section, and site —
so a counts table carrying a dimension the design does not model was
read as repeated units. That is exactly what
prep_counts_daily_effort() produces: it emits one row per
(date, strata, effort_type), and bank and boat counts on
the same day are two units, not one day counted twice.
With count_time_col supplied, the consequence was a
wrong number and no warning. All rows for a day collapsed into one, so
the effort types were averaged rather than summed and
the surviving row kept the first row’s label: a four-day example whose
true total is 121 angler-days reported 60.5, and the rows that vanished
were labelled bank. With k effort types the
estimate was off by a factor of k.
Inference is kept as the default, so existing correct code is
untouched, but it can no longer fail quietly: where the unit is
ambiguous the call now aborts. This is the third appearance of one root
cause — the key omitted section (#155), then
effort_type (#162) — which is why the fix stops enumerating
dimensions and lets the caller state the unit instead.
creel_schema() gains site_col and
circuit_col (#126). A bus-route interview has to name the
site and circuit it was taken at, or add_interviews()
cannot join the site inclusion probability — but the schema had no way
to say which source columns hold them, so the connect layer dropped them
and the join aborted with an error that pointed nowhere near the cause.
Both default to NULL; nothing else changes.
The second major bump. It closes one defect class opened by the 2026-08-14 seam audits: a parameter estimated from data, then consumed as though it were known. Six issues (#135, #137, #138, #139, #157, #158) turned out to be one bug wearing six hats — a visibility correction, an angler-to-people ratio, a camera calibration, a harvest rate, a reporting rate, and an imputed count were each divided or multiplied into an estimate while contributing nothing to its standard error. Every one of them produced a plausible number and no warning.
Read the Breaking changes section before upgrading. Aerial and camera designs must now state their corrections or they abort, and several standard errors move upward — including some that were previously smaller than the single term they had omitted.
Aerial designs must supply
visibility_correction and
angler_ratio. Both arguments previously defaulted
silently to 1.0. Not supplying a correction is not the same claim as
declaring that none applies, and only one of those should be silent. To
declare that none applies, pass the string "none": the
point estimate uses 1 and the corresponding standard-error component is
reported as NA, never 0, because a zero is
indistinguishable from a term that never propagated. To assert instead
that a multiplier is known exactly, supply its standard error as 0
deliberately
(visibility_correction = 1, visibility_se = 0).
est_effort_camera() without
interviews must pass calibration = "none", and
then reports NA standard error. Expanding a raw
camera count by h_open alone assumes each counted object
contributes exactly one angler-hour per hour open — a calibration of 1
that was never measured. Reaching that path now requires saying so. The
calibration component becomes present-and-unknown rather
than absent, because the correction genuinely applies and simply was not
measured. COMP-05 asserted the opposite and is inverted
deliberately, with the reason recorded in the test.
Breaking (numeric): estimate_effort_aerial()
standard errors move upward wherever a boat count was expanded by
derive_angler_count(). The function never called
compute_expansion_var_contribution(), so a count carrying a
party_size_se reached svytotal() with its
multiplier’s uncertainty discarded — the carrier columns survived
add_counts() and were simply not read. Measured on a
fixture, the dropped component was 560 against a reported standard error
of 236: the missing term was larger than the entire standard error being
reported.
Breaking (numeric): impute_camera_counts()
returns a different object when m > 1. It now
yields a camera_imputations object of m
completed data sets rather than one. m = 1 is unchanged and
still returns a plain data frame.
Breaking (numeric): estimate_mr_harvest()
confidence intervals are no longer built by scaling the endpoints of the
abundance interval when the harvest rate is estimated. That
identity is exact only while the rate is a known positive constant; once
it is estimated the endpoints are themselves random. An estimated rate
now falls back to a symmetric interval built from the full product
standard error.
visibility_correction gains
visibility_se (#135). The correction is estimated from
paired air-ground counts and the standard field method reports its
standard error as routine output (Smucker et al. 2010, eq. 6–7);
tidycreel had no argument that could accept that number. The delta term
E * se_v / v is added once at the total, never per
stratum: v is a shared multiplier, perfectly
correlated across flights, and summing it per stratum in quadrature
would treat it as independent and understate it. On the GLMM bootstrap
path v is resampled once per replicate, outside the model
refit, for the same reason — drawing it per flight would shrink its
contribution like 1/sqrt(n_flights).
Aerial designs gain angler_ratio and
angler_ratio_se (#158). Smucker et al. (2010) apply
two corrections to a raw observer count — a visibility
correction and an angler-to-people ratio of 0.404 — and tidycreel
implemented only the first, overstating shore effort by roughly 2.5×.
The two push in opposite directions (0.404 down, 2.69 up), so applying
only the visibility correction is not conservative: it is biased in the
direction of the correction that was kept.
estimate_mr_harvest() gains
harvest_rate_se (#138). It computed
se_H <- harvest_rate * se_N, which is
product_total_variance() with r_se = 0 — the
package already implemented Goodman (1960) and made it the default in
all three creel-estimates-total-*.R files; this function
simply never called it. Rasmussen et al. (1998) draw the distinction in
the package’s own cited literature: the subtractive form is for terms
“estimated from a sample”, and differs from the population formula “used
when the terms in the product are known, not estimated”. The bootstrap
path now draws the rate once per replicate rather than holding it
fixed.
estimate_exploitation_rate() gains
reporting_rate_se and the third delta term
(u/lambda)^2 var(lambda) (#139), since
d(u)/d(lambda) = -u/lambda. It enters once at the
total: lambda is a single estimate dividing every stratum, so
adding it per stratum and summing in quadrature would treat a shared
divisor as independent. On the stratified path it is applied to the
aggregate, deliberately not inside var_u_h.
Its reporting_rate = 1.0 default is
kept, unlike the aerial corrections. It is a visible,
documented default on an exported argument the caller opts into
adjusting, not a value substituted invisibly inside an estimator. That
asymmetry is recorded in the @param text rather than left
to be rediscovered.
Multiple imputation for camera outages (#137).
impute_camera_counts() filled every outage row with the
model’s fitted mean and returned one completed data set. Inside
svytotal() those predictions are indistinguishable from
observations, so the imputation model’s own error was dropped; and
fitted means are smoother than real counts, so the between-day component
shrank as well. The reported standard error was biased downward twice
over. Each of the m completed data sets is now drawn from
the model’s predictive distribution — coefficients
drawn from their sampling distribution, then counts drawn from the
fitted family. Both draws are needed: drawing only the count treats the
coefficients as known, and drawing only the coefficients still yields a
smooth mean where a real count has sampling noise.
est_effort_camera_mi() estimates once per completed
data set and pools by Afrifa-Yamoah et al. (2020) eq. (5): the
within-imputation mean variance plus the (M+1)/(M(M-1))
between-imputation term — the quantity a single completed data set
structurally cannot have. Their factor is Rubin’s (1 + 1/M)
inflation written over the raw sum of squares; MI-04 pins
that the two forms agree. M = 5 follows the paper’s stated
bias-variance balance. Components are reported as
within_imputation / between_imputation so a
reader can see how much of the uncertainty came from imputing.
validate_shared_multiplier() gives the four
shared-multiplier arguments one validation shape — required,
"none" opt-out yielding NA rather than 0,
all-or-none standard error. The rule is documented once there rather
than restated at each call site.
visibility_correction is named a detection
probability and documented as the reciprocal of the published
ground-truthing ratio (#157). Field studies report
r = ground/aerial, which exceeds 1 exactly when the
correction matters (r = 2.69 for shore anglers); tidycreel
wants v = 1/r = 0.372. The > 1 abort branch
now names the conversion, since that is where a reader of the source
paper lands.
The aerial GLMM’s variance composition is documented as
tidycreel’s own reasoning and is not attributed to
Askey et al. (2018). That paper, this estimator’s cited source, was read
in full while specifying this work and contains no visibility correction
and no bootstrap — it does not speak to v at all, and
propagates uncertainty by cross-validation rather than analytically. Its
nAGQ = 0 is likewise not carried over: the paper warns the
option is less accurate and used it only because their data set exceeded
250,000 observations.
Breaking (numeric): the per-section totals from
estimate_total_catch(),
estimate_total_harvest(), and
estimate_total_release() no longer aggregate to the
.lake_total row as though the sections were independent
when one party-size estimate spans them (#145). This is #144 on a second
partition: the sections path builds its frame by hand instead of routing
through the shared stratum helper, so the strata correction never
reached it. A multiplier estimated once and applied across sections is a
single random quantity common to all of them, so its contributions add
before squaring. The lake-row standard error moves
upward on affected designs; the per-section rows and every
point estimate are unchanged.
The structure is now classified against the section
partition rather than the strata, because sections may cross-cut strata
— a group can be nested within strata while spanning sections. One
consequence is visible: a party-size estimate keyed by a stratum (for
example one per day_type) is nested within strata but
straddles sections unevenly, so the lake row now reports
se = NA with a warning rather than a number that quietly
assumed one geometry or the other. As elsewhere in the package, an
unknown standard error is NA, never a zero and never a
plausible substitute.
A sections total now reports the party-size component its
standard error carries, per row, instead of NULL (#145,
completing #134). NULL means the component was never
propagated, and the sections constructor was saying that while its
se demonstrably contained the term.
Breaking (numeric): add_counts()
now keys the sampling unit on the PSU crossed with the section and site,
not on the PSU column alone (#155). Four places needed to know what “the
same unit” means — duplicate detection, within-day aggregation, the
supplied within-day-variance key, and the party-size constancy check —
and they had drifted into three different answers, none of which carried
the section. They now share one psu_key_cols() definition.
Period enters through the strata, which is where this package models it
(strata = c(day_type, day_period)).
A day sampled in two sections was treated as one unit, with three consequences:
58, still labelled with the first section’s name, with
the other section’s rows absorbed into it. This moved the point
estimate — the daily total came out 58 where the truth was 116
— and nothing downstream could detect it, because the result looked like
a clean frame with one section missing.ss_d was dominated by the difference
between sections rather than the spread within a day:
8888 where the true within-section sums of squares were 50 and 2.expansion_se varies within a single PSU and
blaming two derive_angler_count() calls, on a single
coherent call. Under sections the unit is the day within a section, and
each such unit carries exactly one estimate.The CNT-06 warning also stops firing on ordinary multi-section days and now names the key it judged the repeat on. A genuine repeat — the same unit counted twice with no count time — still warns, and two different party-size estimates inside one unit still abort.
Affects designs with sections or sites. Bus-route designs are
untouched: they hold counts in design$bus_route$data, which
never reaches these checks.
The "partial" party-size geometry now returns a
standard error instead of NA (#150). When one party-size
estimate spans some parts of the partition being summed over and another
sits inside one, the combination needs the group-by-part decomposition —
and compute_expansion_var_contribution() was squaring and
summing the group index away before returning, so
add_expansion_covariance() had nothing to combine and
correctly refused. The decomposition is now carried alongside the scalar
component, and the exact combination is
Var = Σ_g (Σ_p rate_p × basis_{g,p} × se_g)²: contributions
from one estimate add before squaring because its error is common to
every part it covers, while contributions from different groups come
from disjoint interview subsets and add as variances.
The "nested" and "shared" numbers
do not move. Both are special cases of that formula, but each
keeps its own arithmetic rather than being re-derived through it, so
their results are unchanged bit-for-bit. Only the case that previously
returned NA produces a new number, and it lands strictly
between the two shortcuts the old code refused to choose between —
quadrature understates it, the linear sum overstates it.
This matters most on the sections path introduced in #145, where
"partial" is ordinary rather than exotic: sections
cross-cut strata, so a party-size estimate keyed by
day_type straddles sections unevenly and forced the
.lake_total row to NA. The refusal is retained
for the case where no decomposition was carried, since a combination
that cannot be computed still must not be guessed.
prep_counts_boat_party() gains
mean_party_size_se, and emits the expansion_*
carrier columns when it is supplied (#143). This function performs the
same boat-to-angler expansion as derive_angler_count(), but
wrote no carriers and had no argument through which a party-size
standard error could be given — so on this path the component was not
merely omitted by default, it was unreachable, and no user action could
recover it. Because this is the pipeline the documentation calls
preferred, the two documented routes to one expansion were not
statistically equivalent and nothing said so: the prep path reported the
pre-3.2.0 understated standard error with
se_expansion = NULL as the only signal.
The emitted basis is boat_count * correction_factor, not
the bare boat count, because this function applies the correction to the
product — a bare basis would be the derivative of a quantity it never
produces and would trip the #131 desync guard. expansion_of
is "daily_effort" for the same reason. Omitting the
argument still leaves the component absent rather than zero.
The creel_error_expansion_basis_desync message now
states that correct hand-rescaling reaches it too (#148).
expansion_of records a column name rather than a scale
factor, so a basis correctly rescaled alongside its count is
indistinguishable from one left behind, and both are refused. Refusing
both remains the conservative and correct choice, but the message
described only the mistake — and every instructional example in the
companion book met it with arithmetic that was right. The wording
changed; the check did not.
Relatedly, the four carrier columns are now documented as
package-written and not user inputs. Overwriting
expansion_of to name a transformed column silences the
guard whether or not the basis was actually rescaled, which re-enables
the defect the guard exists to catch. Use
period_length_col, which scales count and basis together
and can be verified, rather than asserting the rescale.
Breaking (error):
est_effort_camera()’s ratio-calibration path now refuses a
counts table that holds more than one row for the same day, rather than
silently double-counting it (#142). The calibration pairs interview days
to count rows by date membership and reads the day’s effort total once
per matching row, so a repeated date entered
rho = sum(E_d) / sum(C_d) twice on both sides, and the
survey total of raw counts counted it again. This moved the
point estimate, not only the standard error — 16 to 19.5 on the
package’s own five-day test fixture, a 22% shift produced by a
duplicated row carrying no new information. add_counts()
only warns about repeated PSU rows (CNT-06), so such a table reached the
estimator intact.
The table is refused rather than averaged because two counts on one
day are either sub-period snapshots or a data error, and nothing on this
path can tell which. Callers with genuine sub-daily counts should pass
count_time_col to add_counts(), which already
collapses them to one row per day; the error names the offending dates
and says so. The raw-count path (h_open, no interviews) is
deliberately unchanged: expanding a duplicated PSU row through
svytotal() has the same shape in every design, and that is
a wider question than this fix.
Three cases where a quantity that was unknown, malformed, or modelled reached an estimator as though it were observed. All three were found by the statistical seam audits of 2026-08-14; none produced an error, a warning, or an implausible number.
Breaking (numeric):
estimate_total_catch(),
estimate_total_harvest(), and
estimate_total_release() no longer combine a shared
party-size estimate across strata as though the strata were independent
(#144). The stratified total variance adds per-stratum variances because
strata are sampled independently (Pollock, Jones & Brown
eq. 3.12–3.13); a multiplier estimated once and applied to every stratum
is not stratum-independent error, and the covariance the sum omitted is
2 Σ_{h<k} R_h R_k s_h s_k. Standard errors were
understated by up to sqrt(H) on the expansion term for H
strata. This is the default configuration, since
mean_party_size() without by returns one
estimate. Reported standard errors move upward on
affected designs; point estimates are unchanged, and designs whose
party-size estimate is per-stratum are unchanged bit-for-bit. Where
expansion groups straddle strata unevenly the combination is not
recoverable from per-stratum components, so the standard error is
NA with a
creel_warning_expansion_structure_unknown warning rather
than a silently chosen formula. The correction reaches the ungrouped,
grouped, and per-species totals; the per-section path
aggregates its lake row separately and is still affected — see
#145.
The three totals now report the party-size component they carry,
as se_expansion (#134). They routed through the effort
estimators, whose standard error includes the term, but passed no
se_expansion to their constructors — so a totals object
whose se demonstrably contained the component reported
NULL, the value documented to mean “never propagated”.
Anyone applying that test to a total drew the opposite conclusion from
the truth. The reported number is now produced by the same code that
folds the term into the variance, so the two cannot drift apart. The
per-section constructor is not covered; it builds its result frame by
hand and still reports NULL (#145).
print() on a creel_design now shows the
count column and whether the party-size term is carried (#124). Counts
whose expansion carriers were dropped by an ordinary
select() are indistinguishable from counts that never had
them, so the design print is the last point at which the loss can be
surfaced while the user can still act on it. Both lines print whenever
counts are attached. This also closes the older note that the design
never showed which column it used as the count.
tidy() is documented as lossy for uncertainty
components, with the reason: a tibble column cannot hold the
NULL-versus-NA distinction the component
contract depends on. se_between and se_within
are likewise documented as not reconstructing se on
expansion designs.
derive_angler_count() now writes a fourth carrier
column, expansion_of, naming the column the expansion basis
is the derivative of, and add_counts() aborts when the
count column is not that column (#131). expansion_basis is
d(count)/d(party_size), so a count transformed between the
two calls — the documented
mutate(angler_hours = angler_count * shift_hours) pattern,
for one — scales the count and leaves the basis in the old units. The
party-size variance component then came out understated by exactly the
scale factor while remaining present and non-NULL, so it
read as propagated: on a six-day design with a ×12 shift length,
se_expansion was 3 where the same physics expressed through
period_length_col gives 36. Point estimates were
unaffected. Supply the untransformed count and
period_length_col, which scales the count and the basis
together. Breaking: pipelines that premultiplied the
count while retaining the carriers now abort.
mean_party_size() now names its "se"
attribute by the group key, and derive_angler_count()
addresses it by name (#133). The attribute was matched by row order
while the means were joined by key, so any length-preserving reordering
of the lookup — an arrange(), most habitually — gave every
stratum another stratum’s standard error, silently and with the point
estimates unchanged. On a two-stratum design the weekday and weekend
standard errors swapped outright. A by-form lookup whose
"se" attribute has no names is now refused rather than
matched positionally; single-row lookups and the scalar form are
unaffected, having no order to go stale. The
expansion_group attribute was checked for the same hazard
and does not have it: it is built from the counts rows, never indexed
into the lookup.
add_counts() now aborts when counts
carries some but not all of the expansion_* carrier columns
(#132). They are written together by derive_angler_count(),
so a proper subset can only come from partial deletion. The gate
previously required the full set and otherwise took the no-carriers
path, which left an expansion_se sitting visibly in the
table while the party-size variance component silently went missing.
Point estimates were unaffected; se_expansion came back
NULL. Dropping all of them is still undetectable at this
seam — see #124.
Camera ratio calibration reports NA rather than an
exact ratio when a stratum has a single paired interview/count day
(#136). The calibration ratio has no measurable spread from one pair, so
its variance is unknown, not zero; the delta term
T² × var(ρ) previously vanished and the maximally uncertain
calibration was reported with the same standard error as a perfectly
known one. The NA propagates into the combined standard
error and the confidence interval, and a warning names the stratum.
Strata with two or more paired days are unchanged.
The single-day test counts distinct paired dates rather than matched
count rows, so a counts table holding two rows for one date — which
add_counts() only warns about — cannot present one day’s
information as two and restore the false-precision path. The variance
denominator is unchanged, so no existing standard error moves. That such
a table also shifts the point estimate is a separate and older defect,
filed as #142.
Camera effort estimation now warns when the counts carry
.imputed rows (#137), naming how many days contain imputed
counts and what share of the total they are.
impute_camera_counts() flags rows it filled with model
predictions, but nothing downstream read the flag: inside
svytotal() predictions are indistinguishable from
observations, so the imputation model’s prediction uncertainty is
dropped and the between-day variance is further understated because
predictions are smoother than real counts. The reported standard error
is a lower bound. Propagating the prediction variance is still open
under #137.
.imputed now survives within-day aggregation by
collapsing with any(), alongside the existing mean-collapse
for the count and expansion_basis. A day is imputed if any
of its sub-counts was; taking the first sub-count’s value, as every
other column does, let a day whose first count was observed report
itself as fully observed, and the warning above never fired for designs
using count_time_col.
creel_estimates objects now carry
se_components, a named list of the standard-error
contributions that make up se, and print()
reports each one with its relationship to se (#141). The
contract is the one the party-size component has followed since 3.2.0,
generalised: an absent name means the component does not apply to that
path or was never propagated, NA means it applies and is
unknown, a finite value is a contribution and never se
itself, and none of them is ever 0 — a zero cannot be told
apart from a component that never propagated. se_expansion
is unchanged and still supported; the constructor now mirrors it into
se_components[["party_size"]] so a reported component and
the standard error containing it cannot drift apart, which is the defect
#134 was.
Camera effort estimation reports its two delta-method terms
separately as the count_sampling and
calibration components (#141). Since 3.3.0 a stratum with
one paired interview/count day gives its calibration ratio an unknown
variance, which correctly makes the whole standard error NA
(#136) — Var(E) = Σ_h [ρ_h² Var(T_h) + T_h² Var(ρ_h)] is
unknown if any Var(ρ_k) is, and reporting the measurable
part as the standard error would publish a lower bound under the name of
the real thing. That NA stays. What changes is that the
count-sampling half is now reported as a finite component alongside it,
so one thin stratum no longer hides everything that is known.
The reported components reconstruct se exactly, and the
raw-count path omits calibration entirely rather than
reporting it as NA, because that path has no calibration
ratio at all. No standard error changes value.
tidy() remains lossy for these components, for the
reason already documented: a tibble column collapses an absent component
and an unknown one into the same NA.
The sampling error of an estimated party size now reaches the effort standard error (#121). A mean party size taken from interviews multiplies the boat component of every count, so its error is one error applied many times rather than fresh noise per count: it does not shrink as counts accumulate. Treating it as known made every count-expanded effort standard error too small.
mean_party_size() now returns that standard error as a
"se" attribute, and derive_angler_count()
reads it, so the usual pipeline propagates the term with no extra
argument:
counts |> derive_angler_count(
bank = bank_anglers,
boat_count = angler_boats,
party_size = mean_party_size(interviews, n_anglers, angler_type = angler_type)
)Supply party_size_se directly to override it, in any of
the three shapes party_size accepts (scalar, column,
lookup).
The component is reported as se_expansion on the
returned estimates object and is included in se, so it
reaches catch, harvest, and release totals as well. The estimates tibble
keeps its existing seven columns.
est_biomass() can now propagate the length-weight
regression error (#117). a and b are point
estimates from a regression, and a * L^b multiplies every
length bin, so their error is perfectly correlated across bins and does
not shrink as bins are added.
Supply alpha_se, b_se, and L0
together — all three or none:
est_biomass(ld, a = 0.0088, b = 3.1, alpha_se = 0.05, b_se = 0.03, L0 = 250)The allometry is rewritten about a pivot length L0 as
W = alpha * (L / L0)^b, and the delta method applied in
(alpha, b). The parameter covariance is then absent
by construction rather than by assumption: on the raw
(a, b) scale the two are typically correlated below -0.99,
so dropping their covariance there would overstate the variance
severalfold. L0 should be the geometric mean length of the
calibration sample, and alpha_se the intercept SE from a
regression centred there — not the standard error of a.
Reported as attr(x, "biomass_se_params") and included in
biomass_se. Absent — NULL, not 0
— when the arguments are not supplied.
mean_party_size() output, because a
variance component that was previously dropped is now carried. Estimates
themselves are unchanged; only their uncertainty moves. Designs that
pass a bare number or a column as party_size are
unaffected, since no standard error is available for those.When no party-size standard error is available the component is
omitted, not set to zero. se_expansion is
NULL rather than 0, because a zero would
produce a standard error identical to an unpropagated one while
appearing to have been propagated. A party size estimated from a single
interviewed party yields NA, which propagates to an
NA standard error rather than being read as
certainty.
derive_angler_count() writes three further columns —
expansion_basis, expansion_se, and
expansion_group — when a standard error is available.
add_counts() recognises all three and excludes them from
count-column detection, so they cannot make an otherwise unambiguous
counts table look ambiguous.
derive_angler_count() builds the single angler-count
column add_counts() needs from the columns a clerk actually
records. Two forms, matching the two ways boat anglers reach the
form:
# Anglers aboard were counted directly
counts |> derive_angler_count(bank = bank_anglers, boat_anglers = boat_anglers)
# Boats were counted; anglers aboard were not
counts |> derive_angler_count(
bank = bank_anglers,
boat_count = angler_boats,
party_size = mean_party_size(interviews, n_anglers, angler_type = angler_type)
)party_size accepts a single number, a column of
counts, or a lookup table keyed by stratum, so a party size
that differs between weekdays and weekends can be applied per group
rather than averaged away.
boat_count and boat_anglers are separate
arguments deliberately. boat_count counts
hulls, and adding it to an angler total is a units
error that produces a plausible-looking number; requiring
party_size alongside it makes that impossible to do by
accident. Supplying both boat forms is an error, since they are two
routes to the same quantity.
Components are added with na.rm = FALSE: a count that
was not taken and a count of zero anglers are different observations and
stay different.
Until now this derivation was available only on the sampled-day
prep_counts_* seam, via
prep_counts_boat_party(). The raw-count pipeline — the one
that takes a within-day count schedule through
count_time_col and derives the within-day variance
component itself — had no equivalent, so callers there built the total
by hand. Closes #119.
mean_party_size() returns the mean anglers per boat
party from an interviews table, optionally by stratum. It filters to
boat parties, and errors rather than returning NaN when no
row matches — a silent NaN would propagate into every
expanded count.
Bus-route estimators no longer report a confidence bound below
zero. Every ci_lower produced by
estimate_effort(), estimate_total_catch(),
estimate_total_harvest(),
estimate_total_release() and
estimate_harvest_rate() on a bus-route design is now
clamped at zero, in both the ungrouped and by-grouped paths
and in the bootstrap columns (ci_lo_boot). Bus-route was
the last family of estimators in the package without this clamp; the
product totals, exploitation rate and length compliance already had
it.
This changes reported numbers only where the old bound was outside
the parameter space. Angler-hours, fish and fish-per-hour cannot be
negative, so a symmetric Wald bound below zero was never a possible
value for the quantity. It is reached whenever the coefficient of
variation exceeds roughly 0.51 — routine for a bus-route survey with few
sites, unequal inclusion probabilities, or catch concentrated in one
interview. The package’s own bootstrap snapshot fixture was already in
that regime: estimate_total_harvest() reported an estimate
of 115 with an SE of 78.3 and a lower bound of
-38.8, which is now 0. On a deliberately
skewed two-site design with p_site of 0.05 and 0.95 the
excursion is larger, with the total-catch bound moving from
-15999.40 to 0 and the harvest-rate bound from
-54.85 to 0.
A clamped bound of exactly zero means the interval is wide relative
to the estimate. It is not a statement that the quantity could be zero,
and the clamp does not narrow the interval or change the estimate or the
standard error. See ?creel_confidence_intervals. Closes
part of #95.
New topic ?creel_confidence_intervals states the two
conventions the package follows when building intervals: transform where
a principled transform for the quantity exists (logit for exploitation
rate, Sadinle’s transformed logit for mark-recapture abundance, optional
log for product totals) and clamp at the feasible limit otherwise; and
use a t-quantile where an estimator has a design degrees-of-freedom to
appeal to, a normal quantile where it does not. Written down so a new
estimator does not have to pick by coin flip. Closes #95 and
#99.
est_biomass(), est_mean_length(),
est_compliance() and est_mean_age() now record
why they use a normal rather than a t quantile. Their standard error is
propagated from the per-bin standard errors of a length or age
distribution, so there is no local sample size to key degrees of freedom
to: the row count is the number of bins, which is the caller’s binning
choice, and the row totals are expanded estimates rather than counts of
measured fish. Keying a t-quantile to either would make the interval
narrow as bins got finer, with no additional fish measured. Closes
#99.
est_biomass() now states that the length-weight
parameters a and b are treated as known
constants, so biomass_se omits their estimation error and
should be read as a lower bound. Because a * L^b multiplies
every bin, that error is perfectly correlated across bins and does not
shrink as bins are added. Measured on the documented example it adds
roughly 2–11% to a coefficient of variation of 40–65% — minor there, but
material for a survey precise enough to reach a count CV near 10%, or
when a/b are borrowed from a system whose fish
differ in size. Propagating the term needs an API that can accept the
regression’s standard errors and their covariance; tracked in
#117.
Argument guards on truncate_at and
conf_level now reject a value whose length is not 1, rather
than letting it reach the comparison. Passing
truncate_at = c(0.5, 1) to
estimate_catch_rate() raised base R’s
'length = 2' in coercion to 'logical(1)', and
numeric(0) raised
missing value where TRUE/FALSE needed — both of which name
neither the argument nor the constraint it violated. The intended error,
which cites the argument and its default, now fires instead. Affects
estimate_catch_rate(), the bus-route incomplete-trip path,
and est_effort_camera().
A conf_level or truncate_at of
NA_real_ still reaches base R’s “missing value where
TRUE/FALSE needed”. That gap predates this change and is shared by the
six other guards written to the same pattern; it is left for a single
pass over all of them rather than fixed at three sites only.
The first major bump since the package adopted semantic versioning. It closes the dimensional seam audit opened 2026-08-07: 27 findings, ten of them breaking changes to what an estimator returns. Estimates now carry the unit of the quantity they report, derived from the arithmetic the package performed rather than declared by the caller.
Read the Breaking changes section before upgrading — bus-route, ice, instantaneous, aerial and camera designs all report different numbers than 2.5.0 did, because 2.5.0’s numbers were wrong.
Estimates now carry the unit of the quantity they report.
creel_estimates objects gain a unit field,
print() shows a Unit: line,
autoplot() puts it on the y-axis, and
write_estimates() records it in the CSV header. This
replaces hardcoded axis and header strings, which could not tell that
the number underneath them had changed dimension.
The unit is derived, never declared. A unit the
caller types is exactly as trustworthy as the axis label on the poster —
a second place to write the wrong thing — so tidycreel asserts one only
where it performed the arithmetic that produces it: angler-hours on the
count side when add_counts() multiplied by T_d,
angler-hours on the interview side when add_interviews()
multiplied trip hours by a supplied party size, and party-hours when it
did not.
Everywhere else the unit is NA, meaning unknown —
deliberately not “angler-days”. A bare numeric count
column may be an instantaneous head count or effort the caller already
expanded, and example_counts is the latter; guessing
between them would put a confident label on a number that may be in
either unit, which is the failure this machinery exists to prevent. An
absent Unit: line is the claim that tidycreel does not
know, which is a different statement from a default.
est_effort_camera() gains n_anglers,
which makes the ratio-calibration path’s unit derivable instead of
unknown. The calibration ratio is a ratio of sums, so the camera counts
cancel and the estimate inherits whatever unit the interview effort
column holds — angler-hours and party-hours were indistinguishable, a
factor of roughly two apart on the shipped example and reported
identically. Passing n_anglers, either a column in
interviews or a constant party size, makes the function
perform the normalisation itself, which is what earns the
angler-hours label.
Omitting it now warns and names the ambiguity. That warning is only worth raising because the argument exists to answer it: before, it would have reported a gap the caller had no means to close.
The party-size rule is not reimplemented. This path calls the same
exported compute_angler_effort() that
add_interviews() uses, so a party size of zero is refused
at both seams for the same reason, and they cannot drift apart.
n_anglers here takes a column name or a constant
rather than a tidyselect symbol, matching its neighbouring
effort_col and intercept_col
arguments.
Unit propagation now reaches the rate and total estimators
outside the standard CPUE spine. Species, sections, grouped, bus-route
and regression rates carry fish/<denominator>;
species, sections and bus-route totals carry fish. These
paths reach different constructors than the ungrouped ones, which is why
they were still reporting NA after the first pass.
NA is not a neutral default: it reads as “tidycreel does
not know what this number is”, and it suppresses the unit from
print(), autoplot() and the CSV header, so the
number travels bare. Saying NA where the package does know
is as much a false claim as guessing.
The denominator is a property of the interviews rather than of which rate was asked for, so every rate estimator on one design now reports the same one — asserted between estimators in the tests rather than against a hardcoded string, since a wrong constant can satisfy a literal but cannot make two independent estimators agree.
Visible change: autoplot() y-axis labels on these paths
now read e.g. “Total Catch (fish)” where they previously read “Total
Catch”.
Unit propagation now covers the effort family, where the same quantity is derived three different ways and so takes its unit from three different places.
Bus-route effort reports the interview denominator,
not the count side: E_hat = sum(e_i / pi_i) is built
entirely from interview contributions, so labelling it from the counts
would assert a provenance the number does not have. Aerial effort is
angler-hours unconditionally — an aerial design refuses
period_length_col, which makes h_open the sole
period source.
Camera effort splits by path. The raw-count path is angler-hours for
the same reason as aerial. The ratio-calibration path is
NA: its ratio carries the unit of the
effort_col column in a caller-supplied data frame, which
nothing normalises by party size, so angler-hours and party-hours are
indistinguishable there. Unknown is the honest answer, and the same one
add_counts() gives a bare count column.
estimate_angler_trips() and
estimate_effort_per_acre() now carry units, and both
inherit rather than assert them. These two take a
creel_estimates rather than a design, so they cannot ask a
design what anything is in; each transforms a quantity whose unit it was
handed.
Trips are effort divided by mean trip length, and the divisor is
hours per trip, so the count comes back in whichever actor the effort
was measured in: angler-hours give angler-trips,
party-hours give party-trips. The method name is
"angler-trips" for every caller, which is precisely why the
unit cannot be read off it — a bus-route design with no
n_anglers produces a party-level count that a fixed label
would have reported as angler trips.
Effort per acre composes its unit from the effort’s, keeping
party-hours/acre distinguishable from
angler-hours/acre. An unknown effort unit stays unknown
through both: dividing an unknown quantity does not make it known, and
"NA/acre" would read as a real unit on a plot
axis.
Unit propagation now reaches the mark-recapture and
exploitation-rate estimators, the last group without units, and the
honest answer for most of them is NA.
estimate_exploitation_rate() reports
"proportion" on both the stratified and unstratified paths.
It is the one estimator in the package whose unit no input can change:
divides fish by fish twice, so both actors cancel for every design.
estimate_angler_n() reports NA,
not "anglers". Its M,
n and m arrive as bare numerics that nothing
inspects, and the arithmetic divides counts by counts, so carries
whatever actor the marking protocol marked — anglers on some surveys,
boats or parties on others. Asserting "anglers" would
restate the function’s name rather than derive anything.
estimate_mr_harvest() inherits that unknown for the same
reason: its product is in fish only if counted anglers.
estimate_total_catch(),
estimate_total_harvest() and
estimate_total_release() abort with class
creel_error_unit_mismatch when the effort unit and the
rate’s denominator are both known and disagree. Their product is not a
catch.
Two seams are deliberately excluded. A per-party-hour rate meeting
angler-hour effort keeps warn_party_hours_product()’s
existing warning rather than becoming an error, since that would break
every caller who omits n_anglers. An unknown effort unit is
reported by the T_d warning below rather than a second message, so one
defect produces one diagnosis.
day_length() computes hours between sunrise and
sunset for a latitude and date using the CBM model of Forsythe et
al. (1995). Closed form — no lookup table, no network access, no
location database. Only latitude is needed: longitude and time zone
shift when sunrise and sunset occur, not the interval between them.
horizon selects the depression angle, by name
("sunset", "civil", "nautical",
"astronomical") or in degrees. Days inside the polar
circles saturate at 0 or 24 hours rather than returning
NaN.
Day length is astronomical and is not the same quantity as the
estimators’ , which is the period the counts were randomised within — a
property of the survey design, set by regulation, access hours, or field
protocol. Use day_length() for simulation and planning;
pass the period your protocol actually used to
add_counts().
simulate_creel_data() gains lat and
daylight_hours, either of which adds
daylight_hours and angler_hours columns to the
simulated counts table. lat derives the daily period per
date via day_length(); daylight_hours sets it
directly, as a scalar or a named monthly vector, for surveys whose
fishing day is fixed by regulation. Supplying both is an error.
Supplying neither leaves both columns off, so the default output is unchanged. There is no honest default latitude, and substituting one would put a plausible number where the caller gave none.
add_lengths() warns when a binned release row
carries a fractional count. The guard’s own error message
had always said “a positive integer count” while nothing checked
integrality, so count = 3.5 was accepted silently and
reached estimate_length_distribution(), which aggregates
that column as a per-bin fish count. A fraction of a fish then entered
the distribution and every proportion computed from it.
Warned rather than rejected, matching how n_anglers
treats the same category error: a fractional count of discrete things
signals the wrong column was supplied, not that the data are unusable,
and aborting would break tables that have always been accepted. The
NA message now says “a positive count; non-integer values
warn”, so what it claims and what it enforces agree.
estimate_effort() warns, once per session, when an
instantaneous design carries no period_length_col. Without
T_d the estimator expands the count column to the season and returns it,
which is not angler-hours. The warning states the reading rather than
asserting the unit: tidycreel cannot tell an instantaneous head count
from a column that already holds angler-hours, since both arrive as a
numeric column, so it says that if the column is a count the
result is in angler-days. Numbers are unchanged for these callers.
The three product totals raise the same warning. They call
estimate_effort_total() directly rather than
estimate_effort(), so without this a caller who only ever
asks for a total never heard that the count column had no T_d
applied.
Output from the prep_counts_*() helpers is exempt. That
seam resolves counts into sampled-day effort before
add_counts() sees them, so there is no instantaneous count
left to expand and no T_d to ask for — warning there would fire on the
documented preferred workflow. The marker is carried as an attribute, so
a table piped through intervening dplyr verbs degrades to “unknown”,
which is the safe direction.
estimate_total_catch(),
estimate_total_harvest() and
estimate_total_release() now accept
by = species on bus-route and ice designs, and answer on
the Horvitz–Thompson path. All three resolved by against
the interview columns, which carry no species column, so the call
aborted with Column `species` doesn't exist on both design
types — six combinations, none of them reachable. The species-level
total estimators they would otherwise have reached are stratum product
sums built on the standard interview survey, so routing there instead
would have reproduced the previous entry’s defect in the totals: a
species total contradicting the all-species total on the same
object.
The falsifier is the same partition identity, and it is exact for a
Horvitz–Thompson sum because that sum is linear in its numerator. All
six combinations now satisfy it, and each species’ total over the HT
effort equals that species’ rate to machine precision — the cross-check
tying the totals to the rates. The reported method names the estimator
and the quantity (ht-total-release-species).
use_trips = "all" is still rejected: the completed-trip
guard runs ahead of the species branch, because an incomplete trip
contributes catch-so-far under a completed trip’s inclusion probability
whether or not the numerator is one species.
Species-level rates (by = species) now take the
Horvitz–Thompson path on bus-route and ice designs.
estimate_cpue_species() and its harvest and release
siblings build a per-species interview table and hand it to the
standard interview-survey estimators, so on these two design
types they ignored .pi_i and .expansion — the
defect the previous entry removed from the all-species rates, one
estimator over. Fixing the all-species side first is what made it
visible: one design object then returned both answers, each under a
method string naming the same quantity.
The falsifier is a partition identity rather than a reference value. Species partition the catch and every species shares the same effort denominator, so the species rates must sum to the all-species rate exactly. Before the fix the species sum matched the standard-path rate to the last digit:
| design | rate | all-species | species sum | gap |
|---|---|---|---|---|
| bus-route | CPUE | 0.748339 | 0.937805 | +25.32% |
| bus-route | RPUE | 0.421378 | 0.494953 | +17.46% |
| ice | HPUE | 0.919685 | 0.862944 | −6.17% |
| ice | RPUE | 0.909720 | 0.964467 | +6.02% |
| ice | CPUE | 1.829405 | 1.827411 | −0.11% |
All five now reconcile exactly. Per species the estimator repoints
the numerator at that species’ counts and delegates to the bus-route
estimator the all-species rates already use, so the two can no longer
drift apart, and the reported method gains the -species
suffix on both trip paths (ratio-of-means-rpue-species
where the standard path still reports ratio-of-means-rpue).
use_trips = "diagnostic" is refused with species grouping:
the diagnostic pair returns two estimates per species, and returning
either half under one label is the mislabelling this release is
removing.
Also fixes a regression introduced by the previous entry: that
dispatch resolved by against the interview columns, where
there is no species column, so by = species aborted on ice
designs where it had previously worked, and on bus-route designs where
it had never worked.
Breaking: every species-level rate on a bus-route or ice design moves.
The three rate estimators now dispatch to the Horvitz–Thompson
path on ice designs as well as bus-route ones, and
estimate_catch_rate() gains the bus-route dispatch it never
had. estimate_effort() and all three totals already treated
ice as the degenerate bus route it is documented to be; the rate
estimators were the outliers, so a single design object returned a rate
that its own totals contradict. Both paths reported the same
method string, so nothing in the returned object
distinguished them.
A ratio of HT totals must equal total ÷ effort exactly, which is what says which of the two answers was wrong rather than merely that they differed:
| design | rate | before | totals imply | after |
|---|---|---|---|---|
| ice | HPUE | 0.514328 | 0.478561 | 0.478561 |
| ice | CPUE | — | — | reconciles exactly |
| bus-route | CPUE | 0.466438 | 0.433603 | 0.433603 |
Ice designs consequently take the bus-route use_trips
set — "complete", "incomplete",
"diagnostic" — instead of the standard path’s, so they now
accept the two values their own design type is built on and reject
"all", which is not an estimator on this path. For
estimate_catch_rate() the roving auto-route to
"all" + MOR does not apply on these designs.
Breaking: ice HPUE, ice RPUE, ice CPUE and bus-route CPUE all move.
estimate_harvest_rate() and
estimate_release_rate() now validate use_trips
on the bus-route path. The bus-route dispatch runs before the standard
path’s check and handed the string straight to the estimator, which
branches on "diagnostic", then "complete",
then "incomplete" with no final else — so an
unrecognised value reached the complete-trip code with the trip-status
filter switched off and returned the all-trips answer under the
complete-trip method string, silently. The dangerous input was not a
nonsense string but a valid value typed with the wrong case: on
a fixture of four complete and four incomplete trips,
"Complete" returned 2.816514 over all eight rows where
"complete" returns 2.642202 over four. The standard path
rejected the same input, so whether a typo aborted depended on the
design type.
The valid set on the bus-route rate path is "complete",
"incomplete" or "diagnostic", as documented.
It is deliberately not the standard path’s set:
"incomplete" is a legitimate rate here (Hoenig et al. 1997)
and is not offered there, and "all" is legitimate there and
is not an estimator here, because pooling the two kinds of trip applies
the complete-trip ratio of Horvitz–Thompson totals to numerators that
are catch so far. Matching is exact — "comp" is an error,
not "complete".
The product totals now warn when the rate and the effort they
multiply are in different units. Without n_anglers,
add_interviews() leaves .angler_effort equal
to the raw effort column, so every rate is fish per party-hour
while count-derived effort is angler-hours; both operands are
individually correct but the product is not, unless every party is a
single angler. add_interviews() informed at construction,
but design$angler_effort_col was
".angler_effort" either way, so nothing downstream could
tell the two apart and nothing spoke up where the units actually
collide. Designs now carry n_anglers_supplied, and
estimate_total_catch(),
estimate_total_harvest() and
estimate_total_release() warn on the product path when it
is FALSE. Bus-route and ice designs are unaffected: their
totals are Horvitz–Thompson sums over interviews with no rate
multiplication. The package’s own examples now pass
n_anglers (#112).
estimate_total_release() and
estimate_release_rate() had no bus-route dispatch, so on a
bus-route or ice design they ran the count-based product path and
ignored the inclusion probabilities entirely. The interview-derived
release counts were divided by a svytotal() over count rows
— a different effort basis from the one estimate_effort()
reports for the same design, with no warning. On a fixture whose catch
records set the released count equal to the harvest column interview by
interview, so that the true release total equals the true
harvest total, estimate_total_harvest() returned 465.4 and
estimate_total_release() returned 51.1; the two now agree
to machine precision. Bus-route designs carrying no counts aborted
demanding add_counts(), which they do not need.
estimate_total_release_br() had been correct and
unreachable since it was written (#110).
estimate_release_rate() on a bus-route design
reaches the same estimators as estimate_harvest_rate().
use_trips accepts "incomplete" — the
truncated, Hájek-weighted mean of ratios of Hoenig et al. (1997),
reported as method = "mean-of-ratios-rpue" — and
"diagnostic", alongside the existing complete-trip ratio of
Horvitz–Thompson totals (method = "ratio-of-means-rpue").
Both are releases per angler-hour (#110).
prep_counts_daily_effort() and
prep_counts_boat_party() emitted n_counts and
within_day_var columns that add_counts() never
read, so a within-day variance component supplied through the documented
preferred seam was silently dropped and the reported SE omitted it
entirely — biased downward, the dangerous direction. On
an eight-day fixture with three counts per day the prep seam reported SE
6.93 where the equivalent add_counts(count_time_col = )
route reported 9.52. add_counts() now reads both columns
into design$within_day_var, and the two seams agree exactly
(#109).
The columns are also rescaled into daily_effort squared
units on output — by correction_factor^2, and additionally
by mean_party_size^2 in the boat path.
daily_effort is scaled by those factors but the sum of
squares was passed through untouched, so wiring the slot up without
rescaling would have left the within-day term a factor of
cf^2 away from the between-day term it is added to.
within_day_var is now documented unambiguously as a
sum of squares, not a variance: the estimator supplies
the divisor itself, forming
sum(ss_d) / (n_sampled * (k_bar - 1)), so a variance
understates the component by a factor of k_d - 1. To make
that contract enforceable, within_day_var now requires
n_counts, must be non-negative, and must be 0
wherever n_counts is 1. Supplying the component through
both the columns and add_counts(count_time_col = ) is an
error rather than a double count. Counts tables carrying neither column
are unaffected.
estimate_angler_n() now defaults to Sadinle’s (2009)
0.5 transformed logit confidence interval on the Chapman and Petersen
branches, via a new ci_method = "logit". Every
Chapman and Petersen bound moves. Pass
ci_method = "delta" to reproduce the previous symmetric
Wald interval exactly. estimate_mr_harvest() inherits the
change, rebuilding its interval from the same capture table.
The Wald interval is symmetric while is a ratio with a small integer
denominator and is strongly right-skewed, so it leaves the parameter
space in the regime Chapman exists for. At M = 200,
n = 50, m = 3 it reported
ci_lower = -2124.8; at m = 5 it reported
48.7 against 245 individuals actually observed. Evans et
al. (1996) measured Wald coverage failing on one side 27.9% of the time
against a 2.5% nominal rate.
Sadinle compared nine intervals and found the 0.5 transformed logit “the best of the intervals reported here”, with near-nominal coverage even for small populations and capture probabilities near 0 or 1, where profile-likelihood and Monte Carlo intervals both degrade. Its lower limit cannot fall below the number of individuals observed. It is closed-form, deterministic, and adds no dependency.
m |
before | after |
|---|---|---|
| 2 | [-17486.6, 24318.6] |
[1319.1, 14085.6] |
| 3 | [-2124.8, 7248.3] |
[1143.1, 8259.6] |
| 5 | [48.7, 3366.3] |
[903.9, 4211.2] |
| 10 | [406.9, 1454.9] |
[605.4, 1715.0] |
The Schnabel branch is unchanged — it already inverted Poisson quantiles and could not produce a negative bound.
One boundary behaviour is worth knowing: when m == n,
every individual in the second sample was already marked, the estimator
saturates at , and the logit lower limit sits fractionally
above the point estimate because the data imply . Use
ci_method = "delta" if a bound that brackets the point
estimate matters more than coverage.
estimate_mr_harvest() now derives its interval from
the angler-population interval instead of rebuilding a symmetric one, so
a positive angler bound can no longer become a negative harvest bound.
On the ci_method = "delta" path this is not a numeric
change: the old code used the same degrees of freedom and
se_H = harvest_rate * se_N, so its bounds already equalled
the scaled angler bounds to machine precision.
estimate_angler_n(method = "schnabel") now builds
its large-sample confidence interval on degrees of freedom, where is the
number of sampling occasions. It previously used , the recapture total.
Every Schnabel interval with widens; the point estimate
and se are unchanged.
Hansen & Van Kirk (2018) eq. (A.5) uses , as does
fishmethods::schnabel(), the implementation they modified.
The estimator has one observation per occasion regardless of how many
recaptures land in it, so keying df to treats recaptures within an
occasion as independent and understates the interval. On five occasions
with the reported interval was [1504.28, 2665.02] where the
source gives [1388.48, 3127.07] — 33% too narrow.
The se itself was checked against the same sources and
is correct as it stands. Only the quantile changed.
estimate_angler_n(method = "schnabel") now applies
Chapman’s (1952) small-sample correction by default, dividing by instead
of . Every Schnabel point estimate falls, by exactly in
relative terms: 33% at , 1.9% at 52, 0.2% at 500. Pass
bias_adjust = FALSE for the previous form, which is also
what fishmethods::schnabel() computes.
Dettloff (2023) eq. (6) simulated both forms across population sizes
from to . The unadjusted estimator turns biased high at
moderate sample sizes before settling, which propagates into an inflated
estimate_mr_harvest(); the adjusted form’s bias “approaches
zero as the sample size increases without ever becoming positive”, at
lower variance and no cost in large samples. He recommends the adjusted
estimators “in place of the originals in all scenarios”.
The consistency argument is the other half. Schnabel reduces exactly
to Lincoln-Petersen at , so the unadjusted form meant that
method = "schnabel" on two occasions returned the estimator
the package already declines to default to at
method = "petersen" — bias handling depended on how many
occasions had been sampled rather than on the data.
The se moves only through the delta-method Jacobian,
which is evaluated at the reported . shifts by the constant , so is
unchanged and invSE still matches fishmethods.
The Poisson interval () inverts the distribution of rather than centring
on , so its bounds do not move; the large-sample
interval is built around and does.
estimate_angler_n() gains
method = "schumacher", the Schumacher-Eschmeyer regression
estimator, for occasions. It takes the same inputs as
"schnabel" and fits against through the origin with slope ,
giving . The interval is Seber (1982) eq. (4.17) on degrees of freedom,
and bias_adjust (default TRUE) applies
Dettloff’s (2023) eq. (8) small-sample correction. With
bias_adjust = FALSE the point estimate, invSE
and both bounds match fishmethods::schnabel()’s
Schumacher-Eschmeyer row to printed precision, and the formulas were
checked against Seber’s own worked example (Ricker’s red-ear sunfish: =
423, = 0.1935).
Two details differ from the Schnabel branch on purpose. Degrees of freedom are , not : Seber excludes the first occasion because is identically zero when and so “is not strictly a random observation”. And Dettloff’s eq. (8) numerator sums from explicitly — is the one term here that does not vanish at , so occasion 1 has to be dropped by hand rather than by the algebra.
tidycreel deliberately does not implement the “pick the narrower CI” rule. Hansen & Van Kirk (2018) computed both estimators and “selected the mark-recapture estimator that produced the smallest 95% CI”. Choosing the narrower of two intervals after seeing them conditions on the luckier draw, so the reported interval does not have its nominal coverage. Choose between the estimators on design grounds, or report both.
estimate_angler_n(method = "schnabel") no longer
returns ci_upper = Inf when the recapture total is very
small. The Poisson interval divides by the lower quantile , which is
zero for at the 95% level. Hansen & Van Kirk (2018)
eq. (A.4) substitute Ilienko’s (2013) continuous Poisson, , in exactly
that case; it is positive there and yields a finite bound. The
substitution fires only where the discrete quantile is zero — from the
continuous quantile sits just above the discrete one, so the bound stays
monotone across the seam.
The bound is an interpolation, and the warning that announces it is deliberate. It rests on a continuous interpolation of a discrete distribution at one to three total recaptures; it stands in for “the data do not bound this above” rather than measuring anything.
Implementers should note two traps. The quantile lives in the
shape argument of pgamma(), so it must be
root-found — there is no qgamma() call that produces it.
And the source paper’s own worked example is wrong: it
reports the 0.025 quantile at as 0.24 and draws Figure A.1 to match, but
0.24 is qgamma(0.025, shape = 2), a Gamma(2, 1) quantile.
Inverting their eq. (A.4) gives 0.3292. Equation A.4
transcribes Ilienko’s Definition 3.1 faithfully; the example does not.
Tests pin the implementation against Ilienko’s eq. (1) identity with
ppois(), never against the printed example.
estimate_mr_harvest() now keys its Wald interval to
the number of sampling occasions when the input came from
method = "schnabel", matching the change to
estimate_angler_n() above. It read
angler_n$estimates$n, which for Schnabel is , so the
degrees-of-freedom defect fixed in the estimator survived one function
downstream: with five occasions and the harvest interval used where ,
28% too narrow. Schnabel harvest intervals widen;
Chapman and Petersen are unaffected and still use .
add_counts(count_type = "progressive") now
errors when a day’s shift is shorter than
circuit_time, with condition class
creel_error_circuit_exceeds_shift. It previously warned and
then returned an estimate anyway.
The progressive estimator is Hoenig et al. (1993) eq. 3, with the
number of whole circuits in the day. When no circuit completed, so the
count is not a progressive count of that shift and there is nothing for
to expand. generate_progressive_start() already refused
such a design, so the only way to reach the old warning was a hand-built
schedule — precisely the case with no other guard in front of it.
() is unaffected: that is Robson’s (1961) all-day circuit and remains valid.
estimate_mr_harvest(harvest_rate = ) is harvest
per angler, in fish per angler, and is no longer
bounded above. It was documented as the “proportion of anglers that
harvested fish” and guarded to (0, 1].
Those two readings produce different quantities from the same
arithmetic. with a dimensionless proportion is a count of anglers
who kept a fish; the function returns it as
total_harvest, from estimate_mr_harvest(),
with method = "mark-recapture-harvest". The per-angler-rate
reading is the one the output has always claimed, and the one that makes
the product fish.
The (0, 1] guard did more than mislabel — it enforced
the wrong reading. A fishery averaging 1.4 kept fish per angler is
ordinary, and the guard made total harvest unreachable for exactly those
fisheries by erroring on the correct input.
No numeric result changes. Every previously legal call returns what it always did, because the multiplication is untouched. What changes is which quantity you are told to supply, and that values above 1 are now accepted. If you were passing a proportion of anglers, your input was answering a different question than the output claimed to ask, and it should be replaced with mean fish kept per angler.
add_counts() now aborts with class
creel_error_aerial_period_length when
period_length_col is supplied on an aerial design, and
est_effort_camera() aborts with class
creel_error_camera_period_length when its raw-count branch
is handed counts that already carry T_d.
Both estimators already have a period-length term of their own:
aerial scales the count by h_open / v (Pollock Eq. 15.4)
and camera’s raw-count fallback scales by a supplied
h_open. Once add_counts() began applying
period_length_col for any count type (see the previous
entry), a design carrying both multiplied by time twice — on a 4-day
fixture with h_open = 14 and T_d = 2 the aerial total went
from 1400 to 2800, and the unit spine labelled that 2800
“angler-hours”.
This is a regression in the development version only; no released
version applied T_d on those paths, so callers of released tidycreel are
unaffected. Anyone who added period_length_col since that
change should remove it and set the period length through
h_open instead. Camera’s ratio-calibration path is
deliberately unaffected: it divides by mean(count) before
multiplying by count, so a constant T_d cancels out of the
estimate.
add_counts() now applies
period_length_col to instantaneous counts instead of
discarding it. Supplying the column on an instantaneous design used to
be accepted, recorded in design$period_length_col, and then
ignored — the estimate came back as the bare counts summed over days,
with the T_d column left sitting unread in design$counts.
Effort estimates move for anyone who passed it: on an 8-day fixture with
T_d of 8–14 hours the total went from 140 to 1780.
An instantaneous count is a snapshot of how many anglers were present
at one moment, not effort. Effort is that count times the length of the
period it was randomised within, Ê_d = C̄_d × T_d (Hoenig et al. 1993).
The multiplication happens per PSU at attach time, so the ungrouped,
grouped, sectioned and within-day-variance paths all inherit it, and
multi-count PSUs get their ss_d scaled by T_d² so the
within-day variance stays in effort² units.
Applying T_d per date rather than after aggregation is deliberate: the collapsed form computes C̄ × T̄ where the target is the mean of C × T, and the two differ by Cov(C, T). Anglers fish more on long days, so that covariance is positive and the collapsed form biases low. Multiplying per date makes the term exactly zero at any stratum width, which removes the constraint on stratum design that would otherwise follow from T varying within a stratum.
The positive-and-finite check on period_length_col now
runs wherever the column is supplied. It previously lived inside the
progressive-only block, so a zero or negative period passed unchecked on
an instantaneous design.
n_anglers now means a party size, not a tidyselect
column position. It was resolved through
tidyselect::eval_select(), where a bare integer selects a
column by position, so n_anglers = 1L — the
literal in add_interviews()’s own signature — selected
column 1 and multiplied effort by whatever it held. On interviews whose
first column is numeric that silently produced
.angler_effort = hours × <that column>; on the
shipped column order it failed with
* not defined for "Date" objects, naming neither the
argument nor the column it chose. Which of the two you got depended on
your column order. It also set n_anglers_supplied = TRUE,
switching off the warning that exists to catch exactly this
mismatch.
A bare number is now a constant party size:
n_anglers = 1 states that every interview is a single
angler, and n_anglers = 3 that every party held three. Bare
column names are unaffected. This is also the only way to declare a
genuinely solo-angler survey, and therefore to silence the party-hours
warning above without inventing a constant column.
Party sizes are now validated wherever they come from. Zero, negative
and non-finite values abort — a party of no anglers would silently zero
out that interview’s effort — missing values abort as a stated constant
but warn as a column, and non-integer values warn.
compute_angler_effort() follows the same contract; it is
the other exported entry point that writes
.angler_effort.
Bus-route and ice totals now count completed trips
only, in all three quantities.
estimate_total_harvest() already filtered;
estimate_total_catch() and
estimate_total_release() did not, so on one design the
three totals were computed over different row sets and could not be
compared. On a 24-interview fixture split 12 complete / 12 incomplete,
total catch was 1089.81 over 24 rows where the completed-trip figure is
512.31 over 12 — a factor of 2.13.
These are access-point estimators (Malvestuto 1996, §20.3.1.2), and
§20.5.1 builds them by summing completed-trip quantities over
interviews. An uncompleted trip breaks that in two directions at once:
the observed count is catch so far rather than the trip’s
catch, biasing the sum down, while is the inclusion
probability of a completed trip and an uncompleted one is
intercepted with probability proportional to its length (length-of-stay
bias, §20.3.1.1), biasing it up. The two do not cancel
predictably. Incomplete trips support a rate — the truncated Hájek mean
of ratios of Hoenig et al. (1997), reachable via
estimate_catch_rate(use_trips = "incomplete") — never a
total (#112).
estimate_total_catch(use_trips = "all") now
aborts on bus-route and ice designs. It was previously
accepted and silently discarded: "all" and
"complete" returned the same unfiltered number, so the
argument documented as selecting trips did nothing at all on these
designs. "complete" is the default and is unaffected, so
callers passing nothing see no change beyond the completed-trip filter
above (#112).
estimate_angler_trips() and
estimate_effort_per_acre() now reject any
creel_estimates whose method is outside the
effort family ("total", "total-sections").
Both are documented as taking angler-hours but guarded only on class, so
a CPUE object passed straight through: fish per hour divided by hours
per trip, relabelled "angler-trips", no warning. A
fish-valued bus-route total was accepted the same way (#112).
estimate_total_catch(),
estimate_total_harvest() and
estimate_total_release() on a bus-route or ice design now
report method = "ht-total-catch",
"ht-total-harvest" and "ht-total-release"
respectively, in place of the bare "total" all three
returned. "total" is the string the labelling code maps to
effort, so a fish-valued total plotted with a y-axis and title
reading “Total Effort” and exported a CSV whose provenance header read
Method: total — nothing in the returned object said which
quantity it held. On an eight-day bus-route fixture the catch total of
1089.81 fish and the harvest total of 464.77 fish both plotted as “Total
Effort” beside a genuine effort total of 2513.38 angler-hours. The
estimates themselves are unchanged; only the method string and the
labels derived from it move. estimate_effort() still
returns "total", which was correct for it all along.
The ht- prefix names the estimator as well as the
quantity, following the existing product-total-*
convention, so a bus-route Horvitz–Thompson total is no longer
indistinguishable from the standard design’s effort × rate product in
either the object or the exported file (#111).
estimate_release_rate() gains
truncate_at, defaulting to 0.5 hours, with the
same meaning, units, and NULL behaviour it has on
estimate_harvest_rate(). It applies only to the bus-route
incomplete-trip path (#110).
estimate_total_release(design, by = species) and
estimate_release_rate(design, by = species) on a bus-route
or ice design now abort with Column 'species' doesn't exist
rather than returning a number from the standard path. The bus-route
Horvitz–Thompson estimators take no species argument, and
by resolves against the interview table, where a species
column does not exist. estimate_total_harvest() and
estimate_harvest_rate() have behaved this way since their
own dispatches landed; per-species release on these designs was never
estimated from the sampling frame (#110).
estimate_harvest_rate() on a bus-route or ice design
now returns a rate. It dispatched to the Horvitz–Thompson harvest
total of Jones & Pollock (2012) Eq. 19.5 and
returned it with method = "total", so it produced a number
identical to estimate_total_harvest() under a function
documented as returning fish per angler-hour (#107).
Jones & Pollock give bus-route effort and harvest as HT totals
and define no rate estimator, so the rate this design supports is the
ratio of those two totals, H_hat / E_hat — the
ratio-of-means form, and the same quantity and method
string ("ratio-of-means-hpue") the standard designs already
return. The ratio is computed with survey::svyratio()
rather than by dividing two separately estimated totals: the numerator
and denominator come from the same interviews and are strongly
correlated, and treating them as independent overstates the SE by
roughly eightfold on the package’s own fixture.
Grouped results no longer carry a proportion column. A
share-of-total is meaningful for a total and meaningless for a
rate.
estimate_harvest_rate(use_trips = "incomplete") on a
bus-route design now returns a rate. It computed a per-angler ratio,
divided that ratio by the inclusion probability, and summed.
Inverse-probability weights apply to totals, not to ratios, so the
result was neither the population rate nor a total: it grew
linearly with the number of interviews. On a fixture where
every angler harvests at 1 fish per angler-hour it returned 19.2, 38.3,
and 76.7 as the same population was sampled with 4, 8, and 16
interviews. It also dropped the .expansion factor the
complete-trip path applies, and divided by the party’s elapsed hours
rather than angler-hours, so the underlying ratio was fish per
party-hour (#108).
The path now returns the estimator this trip type supports: the
truncated mean of ratios of Hoenig, Jones, Pollock, Robson & Wade
(1997, Biometrics 53:306–317), reported as
method = "mean-of-ratios-hpue". For anglers intercepted
mid-trip they show ratio-of-means weights individual rates by the
square of completed trip length and so “does not provide an
estimate of catch rate that can be used with an independent estimate of
total effort to provide an unbiased estimate of total catch”; the mean
of ratios has the correct expectation. Because interviews are not
equally likely under a bus-route design, the mean is weighted by
.expansion / .pi_i — a Hájek mean rather than the paper’s
plain average — and computed with survey::svyratio() so the
variance is linearised over numerator and denominator together.
estimate_harvest_rate() gains
truncate_at, defaulting to 0.5 hours. The
mean-of-ratios estimator has infinite asymptotic variance,
because 1/L has infinite expectation as trip length
approaches zero; Hoenig et al.
(1997) recommend discarding trips shorter than 30 minutes. The
threshold applies to elapsed trip duration, not to angler-hours — it is
the short clock interval that makes the reciprocal explode, and a large
party fishing briefly clears an angler-hour threshold while still being
the unstable case. truncate_at = NULL disables truncation
and warns. The argument is ignored on every other path, including
use_trips = "complete".
use_trips = "diagnostic" on a bus-route design now
compares like with like. Its two slots held a harvest total and a
quantity that grew with sample size, so the gap read as enormous
incomplete-trip bias when it was a change of physical units. Both slots
now report fish per angler-hour. They remain different estimators —
ratio of HT totals for complete trips, truncated mean of ratios for
incomplete ones — because each is the estimator its trip type supports.
A design carrying only one trip type now aborts with a clear message
instead of failing inside survey with “all arguments must
have the same length”, and the verbose dispatch message
names the estimator actually used rather than always announcing the
complete-trip one.
Bus-route and ice estimate_effort() now return
angler-hours. They read the raw per-party trip duration, so the estimate
was party-hours reported under an angler-hours label — invariant to
party size, and understated by exactly the mean party size in any boat
fishery. They now read the angler-effort column (duration ×
n_anglers) that every other rate estimator already used. On
the same design CPUE is fish per angler-hour, so the old behaviour also
mixed denominators in any effort × CPUE product (#106).
Surveys recording one angler per party are unaffected: with no
n_anglers, angler-effort equals the raw effort, and
add_interviews() already warns. Anything with parties
larger than one will see totals rise by roughly the mean party size. The
ice output column total_effort_hr_on_ice is affected on the
same terms.
add_counts() gains a count_col argument
and no longer picks the count column by position. Previously the count
variable was taken as the first numeric column that was not design
metadata, so a counts table carrying more than one numeric column could
have a row index, a daylight-hours column, or a boat count silently
expanded and reported as “Total Effort” — off by an order of magnitude,
with no warning. When more than one numeric column qualifies,
add_counts() now aborts and lists the candidates; name the
intended column with count_col. Tables with a single count
column are unaffected (#105).
The resolved name is stored on the design as $count_col
and used by estimate_effort(), the sections and grouped
effort paths, the aerial and aerial-GLMM estimators, camera effort,
audit_strata(), and autoplot(), all of which
previously repeated the same positional guess.
Callers of tidycreel.connect::fetch_counts() are
affected: it returns bank_anglers,
angler_boats, and non_ang_boats, so
add_counts() now requires count_col to be
named.
The progressive count articles now state the conditions under
which is unbiased. vignettes/progressive-count-surveys.Rmd
gains a Conditions for an Unbiased Estimate section covering
Hoenig et al.’s
(1993) three requirements — random starting location, randomly chosen direction of travel, and an observer who outpaces the anglers — plus two cautions from the same paper: do not interrupt the circuit to interview, and do not read the count as a number of trips, which “results in a negative bias that can be severe.”
None of these are checkable from the counts table, which is why they belong in prose rather than in a guard.
vignettes/effort-pipeline.Rmd previously derived the
cancellation through an unmotivated that returned the expression to . It
now follows the source’s two-step argument — expand the sampled block by
, then scale by the blocks in the day — which reaches the same formula
and shows why cancels: it defines the blocks the count was scheduled
within, so it has done its work before the estimator runs.
Also corrected: the “circuit time < 30% of ” rule of thumb was not from Hoenig et al. and is not the paper’s condition.
estimate_mr_harvest() attributed its known-constant
harvest rate to Hansen & Van Kirk (2018), which does the opposite:
both factors of that rate are estimated there, given log-normal sampling
distributions, and resampled alongside in the bootstrap behind every
harvest CI. The simplification is this package’s, so the reported
se is a lower bound on the true uncertainty, and
@details now says so rather than crediting a source.
harvest_rate also gains the period it was missing. In
the paper’s the multiplier on the angler population is — days fished per
angler times daily harvest per angler — so the argument must cover the
same period angler_n counts anglers for. “Fish per angler”
alone did not pin that down, and the daily rate is the wrong
one.
estimate_angler_n() documents that its Chapman and
Petersen confidence intervals are symmetric and can fall below zero. is
a ratio with a small integer denominator, so it is right-skewed; at
M = 200, n = 50, m = 3 the
reported ci_lower is -2124.8, and
estimate_mr_harvest() inherits the shape. Chapman is
recommended precisely when recaptures are few, so the docs now direct
small- users to ci_method = "bootstrap", whose percentile
bounds respect the skew. The Schnabel branch already inverts Poisson
quantiles and is unaffected. The interval arithmetic is unchanged in
this release — correcting it moves every shipped Chapman and Petersen
bound.
estimate_exploitation_rate() described
C as a harvest total while pointing at
estimate_total_catch() to produce it. Catch includes
released fish, which were never removed from the tagged cohort, so a
catch total inflates by the release fraction. @param C,
@param strata and vignettes/mark-recapture.Rmd
now point at estimate_total_harvest() and say why. The
estimator is unchanged; only the cross-reference was wrong.
Noted in the docs because no check can catch it: both totals are
counts of fish and both carry unit = "fish", so the
expression is dimensionally coherent. The actor matches and the quantity
does not.
vignettes/flexible-count-estimation.Rmd: the
instantaneous baseline built an open_hours column that no
tidycreel function reads, so the example looked like it accounted for
the length of the fishing day while reporting 135 where its own stated
formula gives 1350 — a 10x understatement in the vignette teaching this
exact topic. The inert column is removed and the units of the
instantaneous estimate (angler-days, not angler-hours) are now stated
explicitly (#113).
vignettes/progressive-count-surveys.Rmd: the
“Multiple Periods per Day” example built open_hours and
shift_hours and passed neither, so it demonstrated the
instantaneous multi-count path inside the progressive article. Both
inert columns are removed and the text now says the chunk shows the
within-day variance decomposition only, without the progressive
T_d expansion (#113).
Inert open_hours calendar columns are removed from
the six remaining places they appeared —
vignettes/progressive-count-surveys.Rmd,
vignettes/effort-pipeline.Rmd and
vignettes/temporal-extrapolation.Rmd.
creel_design() reads only the date and the strata, so the
column was never consulted anywhere it was written. The progressive
article additionally listed the calendar’s open_hours as
the the estimator applies; the real travels with the count data and is
passed as period_length_col (#113).
vignettes/tidycreel.Rmd: two reported values had
drifted from what the chunks print. The total effort estimate is 372.5
angler-hours, not the 358 claimed, and the grouped estimates are 201.9
weekend / 170.6 weekday, not 250 / 108. The grouped comparison now notes
that the calendar holds 10 weekdays to 4 weekend days, so a weekend
total 18% higher is a per-day rate about three times higher. The article
also called example_counts “instantaneous count
observations” when the column holds angler-hours already accumulated
over the day (#113).
example_counts and
example_sections_counts documented their
effort_hours column as an instantaneous count of
angler-hours, which is two different quantities at once. Both now
state that the column holds angler-hours, and that
estimate_effort() expands whatever column it is given
without converting units — raw counts in, angler-days out
(#113).
vignettes/glossary.Rmd sanctioned the same ambiguity
by defining count data as “the observed angler count or angler-hours”.
It now states that both are accepted, that no conversion happens, and
which unit each choice returns (#113).
vignettes/ice-fishing.Rmd described
estimate_total_catch() as CPUE times effort over all
interviews. On ice designs it is a Horvitz–Thompson sum with no CPUE
term and no effort term, over complete trips only — 60 of the vignette’s
72 interviews — and use_trips = "all" is refused. The
standard error comes from Taylor linearization, not the delta method the
text credited (#113).
generate_progressive_start() schedules randomised
circuit start times for progressive count surveys following Hoenig et
al. (1993). Two strategies supported: "discrete" (start
drawn from valid τ-aligned offsets; avoids mid-day bias from the common
U[0, T−τ] error) and "wraparound" (start drawn
from U[0, T) with wrap detection). Returns a
creel_schedule with circuit_start,
circuit_end, is_wrapped, and
direction columns.add_counts() with
count_type = "progressive": multi-circuit designs (multiple
counts per day via count_time_col) were previously blocked
with an error. Now supported — daily effort is estimated as
mean(C_k) × T_d across circuits.
add_counts() multi-circuit progressive: within-day
variance ss_d was in count² units but
compute_within_day_var_contribution() requires effort²
units. ss_d is now scaled by T_d² per PSU
before the progressive effort computation, correcting variance estimates
for multi-circuit designs.
add_counts() progressive:
period_length_col was incorrectly included in the numeric
column scan used to auto-detect the count variable, causing it to be
misidentified as the count. Now excluded from the scan.
simulate_creel_data(): minimum trip effort floor
raised from 0.05 h to 0.1 h to reduce implausibly short simulated
fishing trips.
est_age_distribution() estimates proportional age
structure with SE and confidence intervals from age-frequency interview
data, fully integrated with the creel_design workflow.
Stratified and grouped estimation supported.
est_mean_age() estimates mean age (± SE, CI) from
structured interview data. Complements
est_age_distribution() for reporting age-structured harvest
results.
example_ages — new built-in dataset of simulated age
observations for use in examples and tests.
estimate_harvest_rate() gains species-level
dispatch: pass a species column and the function routes harvest-rate
estimation independently per species, returning a tidy multi-species
result in a single call.
creel_design() gains open_start
parameter for GLMM aerial designs, allowing the survey window to be
anchored to the count time rather than requiring a fixed open
time.
estimate_total_catch(),
estimate_total_harvest(),
estimate_total_release(): strata with effort but no
interview coverage were silently dropped by an inner join in
compute_stratum_product_sum(), biasing season totals low.
Fixed to warn and retain all effort strata (#Tier1-Bug1).
estimate_angler_trips(): stats::sd() on
a single-interview stratum returned NA, propagating
silently into SE and CI. Guard added for n < 2; emits
cli_warn() and returns NA_real_ for SE so the
point estimate remains usable (#Tier1-Bug2).
estimate_effort(): finite population correction
(FPC) was not applied to the expanded effort svydesign,
causing inflated SE for designs with high sampling fractions. Fixed
(#Tier1-Bug5-adjacent).
optimal_n(): named cost_ratio vectors
were applied positionally instead of by stratum name, producing wrong
allocations when stratum order differed. Zero variance
(all s2_h = 0) and zero total (all ybar_h = 0)
produced silent NaN; both now abort with informative
errors. n_total floored at 1 to prevent degenerate
zero-sample result.
adjust_nonresponse():
method = "calibrate" was accepted and matched but silently
ignored — both methods used direct weight rescaling. Now aborts with an
informative error directing users to survey::calibrate()
directly (#Tier1-Bug4).
adjust_nonresponse() replicate-design path:
svy$scale (a variance formula constant) was multiplied by
mean(wt_multipliers), affecting only variance and using an
average instead of per-observation values. Fixed to scale
svy$pweights per-observation and
svy$repweights row-wise (#Tier1-Bug5).
new_creel_validation():
all(logical(0)) == TRUE caused a 0-row results object to
silently report passed = TRUE. Fixed with
nrow > 0 guard; empty validation results now correctly
return passed = FALSE.
design-validator: ybar_h,
s2_h, and n_proposed were consumed
positionally against named N_h, producing wrong stratum
indexing when order differed. All three now rekeyed by
strata_names before indexing.
validate_incomplete_trips():
perform_tost() crashed or silently passed when
se_diff == 0 (identical SEs) or df <= 0
(n = 1 group). Early- return guards added for both
degenerate cases; isTRUE() used in grouped- passed
aggregation to prevent NA propagating into
if().
schedule_generators(): inclusion_prob
could silently exceed 1 when
p_site * (crew / n_circuits) > 1. Now aborts with an
actionable message.
creel_palette(n): modular recycling used 0-based
index at palette-length multiples, returning NA at those
positions. Fixed to 1-based modular arithmetic.
coerce_schedule_columns(): unconditional
as.integer(period_id) silently coerced character labels
("AM" / "PM") to all-NA,
filtering all downstream rows. Now only coerces when all
non-NA values are numeric strings; character period labels
are preserved unchanged.
compare_variance(): Taylor and replicate SEs were
paired by row position rather than stratum key. If the two estimators
returned rows in different orders, divergence ratios were computed for
mismatched strata. Fixed with a keyed join; group-column detection now
derived from x$by_vars rather than a hardcoded exclusion
list that would misclassify new output columns.
validation_report(),
standardize_species(), hybrid_design(): second
positional string to cli_abort() / cli_warn()
was silently dropped by cli’s argument handling. Merged into single
message or named vector.
est_age_distribution() and
est_length_distribution(): per-group n was
reporting the global interview count
(nrow(design$interviews)) instead of the within-group
count. Fixed to nrow(wide) per group, consistent with
estimate_total_catch() and
estimate_total_harvest().
est_age_distribution() and
est_length_distribution(): left_join() was
called inside the per-group loop against the full interviews table
(constant across iterations). Replaced with match() lookup
and direct column assignment, eliminating repeated dplyr
overhead.
inst/quarto/templates/creel-report/. The template
demonstrates a full season workflow — design, validation, estimation,
and plotting — using tidycreel helpers end to end.estimate_harvest_rate() and
estimate_release_rate() now default to
use_trips = "complete" (previously
use_trips = "all"). For standard (non-bus-route) designs
that supply trip_status, HPUE and RPUE are now estimated
from completed-trip interviews only. This is the statistically preferred
default: incomplete-trip rates underestimate harvest and release when
anglers keep or release additional fish after being interviewed (Hansen
& Van Kirk 2010). The previous all-interview behavior is no longer
the default but remains fully available.
To restore the previous behavior, pass
use_trips = "all" explicitly:
estimate_harvest_rate(design, use_trips = "all")
estimate_release_rate(design, use_trips = "all")Designs without a trip_status column are unaffected (the
argument has no effect). Bus-route designs already defaulted to
"complete" and are unchanged. Closes #69.
tidycreel design, validation, summary, and plotting helpers
end to end.simulate_creel_data() now returns a
$schedule component — a full-season calendar (one row per
season day) with columns date (Date), day_type
(character), and sampled (logical). Pass directly to
creel_design() as the calendar argument for a
complete round-trip simulation pipeline with no manual column
construction. Unsampled days receive a day_type drawn
proportionally from the day_types distribution. Closes
#68.
sim <- simulate_creel_data(params = my_params, day_types = c(weekday = 5/7, weekend = 2/7))
design <- creel_design(sim$schedule, date = date, strata = day_type) |>
add_counts(sim$counts) |>
add_interviews(sim$interviews,
catch = "catch_total", effort = "hours_fished", harvest = "catch_kept",
trip_status = "trip_status", n_anglers = "n_anglers", interview_type = "roving")Note: this changes the return structure from three
components (interviews, counts,
catch) to four (schedule,
interviews, counts, catch). Code
that checks names by position should switch to name-based
access.
simulate_creel_data() day_types parameter
now explicitly documents that the argument must be a named
numeric vector (not a character vector), with a worked
example showing the correct form
c(weekday = 5/7, weekend = 2/7).@examples block expanded with a multi-stratum
simulation and the full round-trip pipeline from
simulate_creel_data() through creel_design(),
add_counts(), and add_interviews().standardize_species(): added custom_codes
argument (named character vector applied as a second AFS-NA pass),
expanded AFS lookup table with Freshwater Drum ("FRD"), and
corrected misleading “supply a custom code map” documentation that
implied a non-existent function argument. Closes #66.estimate_harvest_rate() /
estimate_release_rate(): added use_trips
argument ("all" default, "complete" to
restrict) with cli_inform notice showing trip-status
breakdown. Documented livewell-observable rationale and downward-bias
risk (Hansen & Van Kirk 2010). Closes #65. Future default flip to
"complete" tracked as #69.estimate_catch_rate() now auto-routes roving
designs: when
add_interviews(..., interview_type = "roving") is set and
use_trips / estimator are not explicitly
supplied, the function defaults to use_trips = "all" and
estimator = "mor" (Hoenig et al. 1997), using all
interviewed trips via mean-of-ratios rather than restricting to complete
trips. Access-point designs (interview_type = "access", the
default) are unaffected. Explicit use_trips or
estimator arguments always override the auto-route. Closes
#67.
New use_trips = "all" option for
estimate_catch_rate(): uses every interview (complete +
incomplete) with the MOR estimator. Previously only
"complete", "incomplete", and
"diagnostic" were accepted.
estimate_catch_rate(by = species) returned all-zero
estimates when catch data contained only "harvested" and
"released" rows (no "caught" rows). Fix was in
source since v2.0.0 but the installed binary at the site-library was
stale; reinstalling now picks up the correct aggregation logic. Closes
#64.add_interviews() interview_type parameter
description corrected: now accurately states that "roving"
triggers automatic estimator routing rather than carrying the false
claim that the flag was “stored metadata only”.estimate_catch_rate() use_trips parameter
and Details section updated to document "all", roving
auto-routing, and the access vs. roving distinction.Starting with this release, tidycreel follows semantic versioning (MAJOR.MINOR.PATCH) and names each MINOR release after a fish species native to Nebraska or the Great Plains. v2.1.0 is named for the Sauger (Sander canadensis), a walleye relative common in Nebraska’s large rivers.
estimate_angler_trips() — estimates angler trip counts
(angler days) from effort and mean trip length using Delta Method
variance propagation.estimate_effort_per_acre() — computes effort density
(angler-hours per acre) by stratum from an extrapolated effort estimate
and supplied acreage.summarize_boat_composition() — returns percent angler
boats by month and day type, computed from the angler-boat and
non-angler-boat count columns.summarize_by_zip() — tabulates interview count and
percentage by zip code from the interview zip code column.summarize_by_county() — maps zip codes to counties via
zipcodeR and returns interview count and percentage by county; emits an
informative error when zipcodeR is not installed.cli_abort() sites,
formal lifecycle badges on experimental APIs, a valid
inst/CITATION, and removal of the scales
dependency from the package surface.lubridate from Imports to
Suggests and added runtime install guards at user-facing
schedule entry points.rlang::caller_env() through the top-level
bus-route estimator internals and relocated
get_site_contributions() into the estimation layer to
tighten call-frame quality and layering.@family tags across the exported surface so the
pkgdown reference is grouped by workflow topic rather than a flat
function list.print.creel_design(),
print.creel_estimates_mor(), and
print.creel_schedule().quickcheck-based property tests and generator
helpers covering the highest-value implemented invariants: INV-01,
INV-02, INV-03, INV-04, and INV-06.86.27%, Codecov configuration, and a project target of
85%.estimate_catch_rate() now accepts
estimator = "mortr" for truncated mean-of-ratios (MORtr),
which applies truncate_at as a mandatory threshold and
labels the method "mean-of-ratios-truncated-cpue".estimate_catch_rate() gains a targeted
argument (default TRUE). Setting
targeted = FALSE excludes zero-catch trips before MOR/MORtr
estimation for incidental species workflows.power_creel() provides a unified tidy entry point for
pre-survey sample-size planning, wrapping creel_n_effort(),
creel_n_cpue(), and creel_power() into a
single consistent interface with mode = "effort_n",
"cpue_n", or "power".compare_designs() compares multiple survey designs side
by side from a named list of creel_estimates objects. An
autoplot() method renders a forest plot of point estimates
with confidence intervals.as_hybrid_svydesign() constructs a hybrid access +
roving survey design from combined access-point and roving-route count
data.compare_variance() computes Taylor linearization
vs. replicate (bootstrap or jackknife) standard errors side-by-side for
any creel_estimates object.adjust_nonresponse() applies nonresponse weighting to a
creel_design and records per-stratum diagnostics.est_effort_camera() adds ratio-calibrated
camera/time-lapse effort indexing.est_length_distribution() adds weighted catch-at-length
/ size-structure estimation from attached length data.autoplot.creel_length_distribution() adds a plotting
surface for weighted size-structure estimates.theme_creel() and creel_palette() add
package-standard plot styling.validate_creel_data() adds field-level schema
validation for creel inputs.standardize_species() adds canonical species-code
standardisation helpers.validation_report() adds formatted validation summaries
that can be exported alongside other report-ready outputs.creel_counts_toy and creel_interviews_toy
are now bundled example datasets for examples, tests, and
documentation.inst/rmarkdown/templates/creel-dashboard/.plot_design() now supports multi-strata designs.autoplot() methods now support opt-in
theme = "creel" styling without changing default
behavior.survey:::onestrat message.aerial-glmm vignette downstream
estimation chunk where example_aerial_interviews was paired
with the wrong design object.Imports to support
the autoplot.* methods.Suggests for
the optional report template.summary.creel_estimates() converts any estimate
object to a creel_summary with human-readable column names
(Estimate, SE, CI Lower,
CI Upper, N). Includes
print.creel_summary() and
as.data.frame.creel_summary() methods. Works for effort,
CPUE, harvest rate, total catch, and grouped variants.
flag_outliers() identifies extreme values in a
numeric column using Tukey’s IQR fence (k = 1.5 default).
Returns the input data frame with is_outlier,
outlier_reason, fence_low, and
fence_high columns appended, and emits a cli
summary of flagged rows. Handles n < 4, empty input, and
zero-row data frames gracefully.
ggplot2::autoplot.creel_estimates() produces a
point-and-errorbar plot from any creel_estimates object.
Ungrouped estimates show a single point with confidence interval;
grouped estimates show one point per group level, colour-coded.
ggplot2::autoplot.creel_schedule() produces a
monthly tile calendar from a creel_schedule object. Sampled
dates are coloured by day type (weekday blue / weekend red); unsampled
dates are shown in grey. Multiple months are displayed as vertically
stacked facet panels.
Single-PSU strata now produce a structured, actionable error
instead of an opaque survey:::onestrat message. The error
names the problematic stratum and suggests increasing the sampling rate
or combining sparse strata.
Fixed a bug in the aerial-glmm vignette downstream
estimation chunk where example_aerial_interviews was paired
with the GLMM design (built from
example_aerial_glmm_counts). The chunk now uses the correct
matching dataset (example_aerial_counts +
example_aerial_interviews).
Imports to support
the new autoplot.* methods.generate_count_times() adds three sampling
strategies for allocating interview periods within a survey day: random,
systematic, and fixed-interval. Supports a seed argument
for reproducibility; returns a creel_schedule object
compatible with write_schedule().
The survey-scheduling vignette now covers the full
pre- and post-season planning workflow:
generate_count_times() through
validate_design(), check_completeness(), and
season_summary().
GitHub issue templates now use structured forms with
blank_issues_enabled: false, routing how-to questions to
GitHub Discussions to keep answers searchable for all users.
CONTRIBUTING.md has been rewritten with current
workflow guidance, contribution types, and community norms for the v1.x
release line.
Launched the pkgdown documentation site at https://chrischizinski.com/tidycreel with a custom Bootstrap 5 theme, full function reference index (46 exports + 15 datasets), and a workflow-driven navbar.
Added a GitHub Actions CI/CD workflow to deploy the pkgdown site automatically on every push to main.