Introduction

This vignette for package groupedHyperframe documents the creation of groupedHyperframe object, the batch processes defined for a groupedHyperframe, and aggregations over multi-level grouping structure.

Prerequisite

Package groupedHyperframe requires the development versions of spatstat family of packages.

devtools::install_github('spatstat/spatstat'); packageDate('spatstat')
devtools::install_github('spatstat/spatstat.data'); packageDate('spatstat.data')
devtools::install_github('spatstat/spatstat.explore'); packageDate('spatstat.explore')
devtools::install_github('spatstat/spatstat.geom'); packageDate('spatstat.geom')
devtools::install_github('spatstat/spatstat.linnet'); packageDate('spatstat.linnet')
devtools::install_github('spatstat/spatstat.model'); packageDate('spatstat.model')
devtools::install_github('spatstat/spatstat.random'); packageDate('spatstat.random')
devtools::install_github('spatstat/spatstat.sparse'); packageDate('spatstat.sparse')
devtools::install_github('spatstat/spatstat.univar'); packageDate('spatstat.univar')
devtools::install_github('spatstat/spatstat.utils'); packageDate('spatstat.utils')

Note to Users

Examples in this vignette require that the search path has

library(groupedHyperframe)
library(survival) # to help hyperframe understand Surv object

Users should remove parameter mc.cores = 1L from all examples and use the default option, which engages all CPU cores on the current host for macOS. The authors are forced to have mc.cores = 1L in this vignette in order to pass CRAN’s submission check.

Additional Resources

A development version of package groupedHyperframe is hosted on Github.

devtools::install_github('tingtingzhan/groupedHyperframe', build_vignettes = TRUE)
vignette('intro', package = 'groupedHyperframe')

List of Terms and Abbreviations

Term / Abbreviation	Description	Reference
`attr`	Attributes	`base::attr`; `base::attributes`
`CRAN`, `R`	The Comprehensive R Archive Network	https://cran.r-project.org
`data.frame`	Data frame	`base::data.frame`
`formula`	Formula	`stats::formula`
`fv`, `fv.object`	Function value table	`spatstat.explore::fv.object`
`groupedData`	Grouped data frame	`nlme::groupedData`
`hypercolumn`	Column of hyper data frame	`spatstat.geom::hyperframe`
`hyperframe`	Hyper data frame	`spatstat.geom::hyperframe`
`inherits`	Class inheritance	`base::inherits`
`kerndens`	Kernel density	`stats::density.default()$y`
`matrix`	Matrix	`base::matrix`
`mc.cores`	Number of CPU cores to use	`parallel::mclapply`, `parallel::detectCores`
`multitype`	Multitype object	`spatstat.geom::is.multitype`
`ppp`, `ppp.object`	(Marked) point pattern	`spatstat.geom::ppp.object`
`~ g1/.../gm`	Nested grouping structure	`nlme::groupedData`; `nlme::lme`
`quantile`	Quantile	`stats::quantile`
`S3`	`R`’s simplest object oriented system	https://adv-r.hadley.nz/s3.html
`search`	Search path	`base::search`
`Surv`	Survival object	`survival::Surv`
`trapz`, `cumtrapz`	(Cumulative) trapezoidal integration	`pracma::trapz`; `pracma::cumtrapz`; https://en.wikipedia.org/wiki/Trapezoidal_rule

`groupedHyperframe` Class

The S3 class groupedHyperframe inherits from hyperframe class, in a similar fashion as groupedData class inherits from data.frame class.

A groupedHyperframe object, in addition to hyperframe object, has attribute(s)

attr(., 'group'), a formula to specify the grouping structure

`groupedHyperframe` with `ppp`-hypercolumn

Function grouped_ppp() creates a groupedHyperframe with one-and-only-one ppp-hypercolumn. Multiple ppp-hypercolumns will not be supported in foreseeable future, as we would need to check for name clash in $marks from the multiple ppp-hypercolumns, which is too much trouble.

In the following example, the argument formula specifies

the marks, e.g., numeric mark hladr and multitype mark phenotype, on the left-hand-side
additional predictors and/or endpoints, e.g., OS, gender and age, before the | separator on the right-hand-side
grouping structure, e.g., image_id nested in patient_id, after the | separator on the right-hand-side.

(s = grouped_ppp(formula = hladr + phenotype ~ OS + gender + age | patient_id/image_id, 
                 data = wrobel_lung, mc.cores = 1L))
#> 
#> Grouped Hyperframe: ~patient_id/image_id
#> 
#> 25 image_id nested in
#> 5 patient_id
#> 
#>       OS gender age    patient_id          image_id  ppp.
#> 1  3488+      F  85 #01 0-889-121 [40864,18015].im3 (ppp)
#> 2  3488+      F  85 #01 0-889-121 [42689,19214].im3 (ppp)
#> 3  3488+      F  85 #01 0-889-121 [42806,16718].im3 (ppp)
#> 4  3488+      F  85 #01 0-889-121 [44311,17766].im3 (ppp)
#> 5  3488+      F  85 #01 0-889-121 [45366,16647].im3 (ppp)
#> 6   1605      M  66 #02 1-037-393 [56576,16907].im3 (ppp)
#> 7   1605      M  66 #02 1-037-393 [56583,15235].im3 (ppp)
#> 8   1605      M  66 #02 1-037-393 [57130,16082].im3 (ppp)
#> 9   1605      M  66 #02 1-037-393 [57396,17896].im3 (ppp)
#> 10  1605      M  66 #02 1-037-393 [57403,16934].im3 (ppp)

Function grouped_ppp() has parameter coords which specifies the column name of $x$- and $y$-coordinates in the input data. Default coords = ~ x + y indicates the use of data$x and data$y for $x$- and $y$-coordinates, respectively. Users may use coords = FALSE for data without $x$- and $y$-coordinates. In this case, the coordinates are filled with randomly generated numbers, and the returned groupedHyperframe has a pseudo.ppp-hypercolumn.

(s_a = grouped_ppp(Ki67 ~ Surv(recfreesurv_mon, recurrence) + race + age | patientID/tissueID, 
                  data = Ki67, coords = FALSE, mc.cores = 1L))
#> 
#> Grouped Hyperframe: ~patientID/tissueID
#> 
#> 207 tissueID nested in
#> 200 patientID
#> 
#>    recfreesurv_mon recurrence  race age patientID tissueID         ppp.
#> 1              100          0 White  66   PT00037 TJUe_I17 (pseudo.ppp)
#> 2               22          1 Black  42   PT00039 TJUe_G17 (pseudo.ppp)
#> 3               99          0 White  60   PT00040 TJUe_F17 (pseudo.ppp)
#> 4               99          0 White  53   PT00042 TJUe_D17 (pseudo.ppp)
#> 5              112          1 White  52   PT00054 TJUe_J18 (pseudo.ppp)
#> 6               12          1 Black  51   PT00059 TJUe_N17 (pseudo.ppp)
#> 7               64          0 Asian  50   PT00062 TJUe_J17 (pseudo.ppp)
#> 8               56          0 White  37   PT00068 TJUe_F19 (pseudo.ppp)
#> 9               79          0 White  68   PT00082 TJUe_P19 (pseudo.ppp)
#> 10              26          1 Black  55   PT00084 TJUe_O19 (pseudo.ppp)

Batch Process on `ppp`-Hypercolumn

In this section, we outline the batch process of spatial point pattern analyses applicable to the ppp-hypercolumn of a hyperframe.

Note that these spatial point pattern analyses should not be applied to a pseudo.ppp-hypercolumn, as the $x$- and $y$-coordinates are randomly generated psuedo numbers.

Batch processes that add a fv-hypercolumn to the input hyperframe include

Batch process that adds an `fv`-hypercolumn
Function	Workhorse	Applicable To
`Emark_()`	`spatstat.explore::Emark`	`numeric` marks (e.g., `hladr`) in `ppp`-hypercolumn
`Vmark_()`	`spatstat.explore::Vmark`	`numeric` marks
`markcorr_()`	`spatstat.explore::markcorr`	`numeric` marks
`markvario_()`	`spatstat.explore::markvario`	`numeric` marks
`Gcross_()`	`spatstat.explore::Gcross`	`multitype` marks (e.g., `phenotype`)
`Kcross_()`	`spatstat.explore::Kcross`	`multitype` marks
`Jcross_()`	`spatstat.explore::Jcross`	`multitype` marks

Batch processes that add a numeric-hypercolumn to the input hyperframe include

Batch process that adds a `numeric`-hypercolumn
Function	Workhorse	Applicable To
`nncross_()`	`spatstat.geom::nncross.ppp(., what = 'dist')`	`multitype` marks (e.g., `phenotype`)

Following example shows that multiple batch processes may be applied to a hyperframe (or groupedHyperframe) in a pipeline (|>).

r = seq.int(from = 0, to = 250, by = 10)
out = s |>
  Emark_(r = r, correction = 'best', mc.cores = 1L) |> # slow
  # Vmark_(r = r, correction = 'best', mc.cores = 1L) |> # slow
  # markcorr_(r = r, correction = 'best', mc.cores = 1L) |> # slow
  # markvario_(r = r, correction = 'best', mc.cores = 1L) |> # slow
  Gcross_(i = 'CK+.CD8-', j = 'CK-.CD8+', r = r, correction = 'best', mc.cores = 1L) |> # fast
  # Kcross_(i = 'CK+.CD8-', j = 'CK-.CD8+', r = r, correction = 'best', mc.cores = 1L) |> # fast
  nncross_(i = 'CK+.CD8-', j = 'CK-.CD8+', correction = 'best', mc.cores = 1L) # fast
#>

The returned hyperframe (or groupedHyperframe) has

fv-hypercolumn hladr.E, created by function Emark_() on numeric mark hladr
fv-hypercolumn phenotype.G, created by function Gcross_() on multitype mark phenotype
numeric-hypercolumn phenotype.nncross, created by function nncross_() on multitype mark phenotype

out
#> 
#> Grouped Hyperframe: ~patient_id/image_id
#> 
#> 25 image_id nested in
#> 5 patient_id
#> 
#>       OS gender age    patient_id          image_id  ppp. hladr.E phenotype.G
#> 1  3488+      F  85 #01 0-889-121 [40864,18015].im3 (ppp)    (fv)        (fv)
#> 2  3488+      F  85 #01 0-889-121 [42689,19214].im3 (ppp)    (fv)        (fv)
#> 3  3488+      F  85 #01 0-889-121 [42806,16718].im3 (ppp)    (fv)        (fv)
#> 4  3488+      F  85 #01 0-889-121 [44311,17766].im3 (ppp)    (fv)        (fv)
#> 5  3488+      F  85 #01 0-889-121 [45366,16647].im3 (ppp)    (fv)        (fv)
#> 6   1605      M  66 #02 1-037-393 [56576,16907].im3 (ppp)    (fv)        (fv)
#> 7   1605      M  66 #02 1-037-393 [56583,15235].im3 (ppp)    (fv)        (fv)
#> 8   1605      M  66 #02 1-037-393 [57130,16082].im3 (ppp)    (fv)        (fv)
#> 9   1605      M  66 #02 1-037-393 [57396,17896].im3 (ppp)    (fv)        (fv)
#> 10  1605      M  66 #02 1-037-393 [57403,16934].im3 (ppp)    (fv)        (fv)
#>    phenotype.nncross
#> 1          (numeric)
#> 2          (numeric)
#> 3          (numeric)
#> 4          (numeric)
#> 5          (numeric)
#> 6          (numeric)
#> 7          (numeric)
#> 8          (numeric)
#> 9          (numeric)
#> 10         (numeric)

Aggregation Over Nested Grouping Structure

When nested grouping structure ~g1/g2/.../gm is present, we may aggregate over the

fv-hypercolumn(s)
numeric-hypercolumn(s)
numeric marks in the ppp-hypercolumn

by either one of the grouping levels ~g1, ~g2, …, or ~gm. If the lowest grouping ~gm is specified, then no aggregation is performed.

The returned object of various aggregation functions, aggregate_fv(), aggregate_quantile() and aggregate_kerndens(), is data.frame instead of hyperframe. This is because the aggregated results are stored in matrix-columns, while the hyperframe class does not support matrix-column.

Aggregation of `fv`-hypercolumn(s)

Function aggregate_fv() aggregates

the function values, i.e., the black-solid-curve of spatstat.explore::plot.fv. In the following example, we have
- matrix-column hladr.E.value, aggregated function value from fv-hypercolumn hladr.E
- matrix-column phenotype.G.value, aggregated function value from fv-hypercolumn phenotype.G
the cumulative trapezoid area under the black-solid-curve. In the following example, we have
- matrix-column hladr.E.cumtrapz, aggregated cumulative trapezoid area from fv-hypercolumn hladr.E
- matrix-column phenotype.G.cumtrapz, aggregated cumulative trapezoid area from fv-hypercolumn phenotype.G

afv = out |>
  aggregate_fv(by = ~ patient_id, f_aggr_ = 'mean', mc.cores = 1L)
#> Column(s) 'image_id' removed; as they are not identical per aggregation-group
nrow(afv) # number of patients
#> [1] 5
names(afv)
#> [1] "OS"                   "gender"               "age"                 
#> [4] "patient_id"           "hladr.E.value"        "hladr.E.cumtrapz"    
#> [7] "phenotype.G.value"    "phenotype.G.cumtrapz"
dim(afv$hladr.E.cumtrapz) # N(patient) by length(r)
#> [1]  5 25

Aggregation of `numeric`-hypercolumn(s) and `numeric` mark(s) in `ppp`-hypercolumn

Function aggregate_quantile() aggregates

the quantile of the numeric-hypercolumn(s). In the following example, we have
- matrix-column phenotype.nncross.quantile, aggregated quantile of numeric-hypercolumn phenotype.nncross
the quantile of the numeric mark(s) in the ppp-hypercolumn. In the following example, we have
- matrix-column hladr.quantile, aggregated quantile of numeric mark hladr in ppp-hypercolumn

q = out |>
  aggregate_quantile(by = ~ patient_id, probs = seq.int(from = 0, to = 1, by = .1), mc.cores = 1L)
#> Column(s) 'image_id' removed; as they are not identical per aggregation-group
nrow(q)
#> [1] 5
names(q)
#> [1] "OS"                         "gender"                    
#> [3] "age"                        "patient_id"                
#> [5] "phenotype.nncross.quantile" "hladr.quantile"
dim(q$phenotype.nncross.quantile)
#> [1]  5 11
dim(q$hladr.quantile)
#> [1]  5 11

Function aggregate_kerndens() aggregates

the kernel density of the numeric-hypercolumn(s). In the following example, we have
- matrix-column phenotype.nncross.kerndens, aggregated kernel density of numeric-hypercolumn phenotype.nncross
the kernel density of the numeric mark(s) in the ppp-hypercolumn. In the following example, we have
- matrix-column hladr.kerndens, aggregated kernel density of numeric mark hladr in ppp-hypercolumn

(mdist = out$phenotype.nncross |> unlist() |> max())
#> [1] 354.2968
d = out |> 
  aggregate_kerndens(by = ~ patient_id, from = 0, to = mdist, mc.cores = 1L)
#> Column(s) 'image_id' removed; as they are not identical per aggregation-group
nrow(d)
#> [1] 5
names(d)
#> [1] "OS"                         "gender"                    
#> [3] "age"                        "patient_id"                
#> [5] "phenotype.nncross.kerndens" "hladr.kerndens"
dim(d$phenotype.nncross.kerndens)
#> [1]   5 512

Grouped Hyperframe

Introduction

Prerequisite

Note to Users

Additional Resources

List of Terms and Abbreviations

groupedHyperframe Class

groupedHyperframe with ppp-hypercolumn

Batch Process on ppp-Hypercolumn