MergeKmeans 0.2.0
Robustness
(pre-release stress campaign; see validation/robustness/REPORT.md)
- Univariate data now default to single linkage: the components of 1-D
clusters form a chain that Ward’s linkage cuts unreliably.
- Unconverged Hartigan-Wong K-means solutions (Quick-TRANSfer cap at
large n) are polished with Lloyd iterations so that the stored
components are consistent with the stored centers and
predict() reproduces them.
- Diverging classification EM for the heteroscedastic variants (an
instability inherent in their implied mixing proportions) now returns
the last stable iteration with a warning instead of failing with an
internal error; converged CEM fits return exactly the parameters that
generated the labels, restoring the
predict()
round-trip.
G = "auto" is now computed from the single-linkage
merge sequence of the overlap matrix regardless of the fitting linkage
(the previous Ward-height gaps were unreliable); its limits (range
2:(K-1), spurious suggestions on structureless data) are
documented.
- Fixed:
variant = "HeEC" crashed on univariate data;
plot(fit, what = "tree") failed for K = 2;
Inf values slipped past validation into cryptic downstream
errors (MergeKmeans, predict, chooseK); overlap_map()
crashed on missing values; predict() silently propagated
NA.
- kmeans++ seeding subsamples very large datasets (n > 1e5),
removing the dominant cost of the n = 1e6, large-K regime.
- New documentation: linkage guidance (Ward vs unequal cluster sizes
and 1-D chains), contamination tolerance, heavy-tailed vs skewed
clusters.
New features
init = "kmeans++" is the new default initialization of
the K-means restarts (Arthur & Vassilvitskii, 2007), stabilizing the
large-K component solutions on which the merging is built;
init = "random" restores the previous behavior.
start argument: MergeKmeans() can now
merge a partition computed by any external engine (a kmeans
object, a ClusterR-style list, or a bare label vector), skipping its own
K-means step entirely. This removes the single-threaded
stats::kmeans ceiling for very large datasets.
G = "auto" cuts the merge tree at the largest gap in
merge heights (also available in recut()); when no
G/omega.star is supplied the fit-time message
now reports the suggested G.
K = NULL (new default) selects the number of components
automatically via chooseK().
chooseK() now reports and marks a recommended
K (one grid step beyond the detected elbow) and gained a
cores argument.
cores argument forks the K-means restarts via the
parallel package (POSIX systems).
- broom tidiers:
tidy(), glance(), and
augment() methods are registered when broom is loaded.
User-facing changes
- Variant names are matched case-insensitively and accept
plain-language aliases (“spherical”, “elliptical”, “spherical-unequal”,
“elliptical-unequal”).
omega.star with Ward’s linkage is now an error (it
previously warned and returned a degenerate unmerged partition).
- A warning is emitted when components contain single observations (K
too large relative to n).
- Documentation: scaling guidance, a “when to use what” discussion
(including the recommendation to prefer model-based clustering for
genuinely overlapping Gaussian ellipsoids), and a recipe for merging
mini-batch K-means solutions of massive datasets.
MergeKmeans 0.1.0
- Initial version: DEMP-K merging of K-means solutions (Melnykov &
Michael, 2020) with the four K-means variants (HoSC, HoEC, HeSC, HeEC),
exact pairwise-overlap computations, single/Ward/average/complete
linkages, the overlap map display,
recut(),
chooseK(), pairwise_overlap(), and
print/summary/plot/predict methods.
- All overlap computations validated against
MixSim::overlap() and by independent Monte Carlo
simulation; the paper’s illustrative examples and pen-digits application
reproduced.