Using Rust inside targets

This vignette covers running Rust as targets steps with rextendr / extendr. A Rust step is deliberately different from a Python or Julia one: there is no live interpreter. Instead the #[extendr] functions in your script are compiled into a dynamic library and exposed as ordinary R functions, which an R post-script then calls. For a gentler tour start with vignette("get_started").

Code blocks are illustrative and not executed when the vignette builds (compiling Rust needs a toolchain).

Do you need tar_target_rs()?

Often not, and it is worth being honest about when it earns its keep. Unlike Python and Julia, a Rust step has no live interpreter and no data hand-off to abstract away: rextendr::rust_source() compiles your #[extendr] functions and hands them back as ordinary R functions. For the simple case you can call it directly inside a plain tar_target(), keeping every native targets convenience (automatic dependency detection, inline R, a single file):

rs/scale.rs: one #[extendr] function:

#[extendr]
fn zscore(x: Vec<f64>) -> Vec<f64> {
    let n = x.len() as f64;
    let mean = x.iter().sum::<f64>() / n;
    let sd = (x.iter().map(|v| (v - mean).powi(2)).sum::<f64>() / n).sqrt();
    x.iter().map(|v| (v - mean) / sd).collect()
}
library(targets)

list(
  tar_target(petal_length, iris$Petal.Length),

  tar_target(petal_z, {
    rextendr::rust_source("rs/scale.rs")   # compiles zscore() and puts it in scope
    zscore(petal_length)                   # petal_length is auto-detected as a dependency
  })
)

This is the recommended starting point when your Rust toolchain is already discoverable, which is the usual situation for local and interactive runs. tar_target_rs() is a convenience wrapper that becomes worth it when you want one or more of the following:

If none of those apply, the plain tar_target() above is simpler and gives up nothing. The rest of this vignette documents tar_target_rs() for when they do.

The two constructors

Both forward every targets::tar_target_raw() argument and return a single targets target, except with a tarpolyglot_*() pattern helper (see “Compile the Rust code only once across branches”), where they return a compile target plus the branched target.

These constructors depend on the rextendr R package. As a result, any limitation of rextendr is inherited by the constructors. Reading rextendr’s own documentation is recommended.

How a Rust step differs

Arguments

Argument Meaning
script Path to the Rust file with #[extendr] functions (required).
post_script R script run after compilation; the compiled functions and inputs are in scope. Required for object mode.
inputs Named vector mapping in-step names to upstream targets, e.g. c(x = "value").
output "object" (default) or "file".
dependencies Crate dependencies as a named list, passed to rextendr::rust_source().
features Cargo features, passed to rextendr::rust_source().
profile Build profile (e.g. "dev" or "release").
toolchain Optional rustup toolchain (e.g. "stable-x86_64-pc-windows-gnu"); sets RUSTUP_TOOLCHAIN for the build.

As with the other languages, script and post_script may be a literal path or a tar_target_path("name") reference to track the file.

Installing Rust and rextendr

  1. Install the Rust toolchain with rustup. On Windows, use the GNU toolchain so it matches R’s mingw ABI:

    rustup toolchain install stable-x86_64-pc-windows-gnu
    rustup default stable-x86_64-pc-windows-gnu
  2. Install rextendr and check it works:

    install.packages("rextendr")
    rextendr::rust_source(code = "#[extendr] fn double(x: f64) -> f64 { x * 2.0 }")
    double(21)   # 42

run_rs_step() puts R, cargo, and (on Windows) Rtools on PATH for the build itself, so a Rust step usually works even in a bare crew worker, but the toolchain must be installed.

Object output (iris example)

R/scale_post.R: the compiled zscore() and the input petal are in scope:

# `petal` is bound from inputs = c(petal = "petal_length").
scaled <- zscore(petal)
data.frame(petal = petal, z = scaled)   # last expression = target value

_targets.R:

library(targets)
library(tarpolyglot)

list(
  tar_target(petal_length, iris$Petal.Length),

  tar_target_rs(
    name = petal_z,
    script = "rs/scale.rs",
    inputs = c(petal = "petal_length"),
    post_script = "R/scale_post.R"
  )
)

File output

rs/write.rs: writes a value to disk, the path itself is built in the post-script:

use std::fs::File;
use std::io::Write;

#[extendr]
fn write_value(x: f64, path: String) {
    let mut f = File::create(path).unwrap();
    write!(f, "{}", x).unwrap();
}

R/rs_files_post.R: builds the path, calls the compiled function, then returns the path (file mode needs the post-script’s last expression to be the path(s), not the function’s own return value):

path <- tempfile(fileext = ".txt")
write_value(x, path)
path
tar_target_rs(
  name = rs_file,
  script = "rs/write.rs",
  inputs = c(x = "value"),
  post_script = "R/rs_files_post.R", # returns the path(s)
  output = "file"
)

Crate dependencies, features, profile

Pass extra crates through dependencies (a named list), and build options through features / profile. rs/parse.rs:

#[extendr]
fn parse_sum(txt: String) -> f64 {
    let v: serde_json::Value = serde_json::from_str(&txt).unwrap();
    v["numbers"].as_array().unwrap().iter()
        .map(|n| n.as_f64().unwrap())
        .sum()
}

R/parse_post.R:

parse_sum(txt)
tar_target_rs(
  name = parsed,
  script = "rs/parse.rs",
  inputs = c(txt = "raw_json"),      # e.g. '{"numbers": [1, 2, 3]}'
  post_script = "R/parse_post.R",
  dependencies = list(`serde_json` = "1"),
  profile = "release"
)

Dynamic branching

Like the other constructors, inputs are real dependencies, so pattern branches them (and crew spreads the branches):

list(
  tar_target(groups, split(iris$Petal.Length, iris$Species), iteration = "list"),
  tar_target_rs(
    name = z_by_group,
    script = "rs/scale.rs",
    inputs = c(petal = "groups"),
    post_script = "R/scale_post.R",
    pattern = map(groups),
    iteration = "list"
  )
)

Plain map(groups) here recompiles the crate in every branch: a Rust step has no live interpreter to reuse, so each branch runs cargo from scratch. With many branches that repeated compilation is the dominant cost.

Compile the Rust code only once across branches

tarpolyglot provides dynamic-branching pattern helpers that mirror the targets patterns but compile the Rust crate a single time and reuse it across all branches. Each is named after its targets equivalent:

tarpolyglot helper targets pattern
tarpolyglot_map() map()
tarpolyglot_cross() cross()
tarpolyglot_slice() slice()
tarpolyglot_head() head()
tarpolyglot_tail() tail()
tarpolyglot_sample() sample()

Use one exactly where you would use the plain pattern:

list(
  tar_target(groups, split(iris$Petal.Length, iris$Species), iteration = "list"),
  tar_target_rs(
    name = z_by_group,
    script = "rs/scale.rs",
    inputs = c(petal = "groups"),
    post_script = "R/scale_post.R",
    pattern = tarpolyglot_map(groups),   # was map(groups): now compiles once
    iteration = "list"
  )
)

On a Rust step, tarpolyglot_map() expands that single constructor call into two targets:

You still write one tar_target_rs(); the companion <name>_rust_lib target appears automatically in the pipeline (visible in tar_visnetwork()), and targets recompiles it only when the Rust source changes. So pattern = tarpolyglot_map(x) turns N x compile into 1 x compile plus N near-instant reloads.

The other pattern helpers behave the same way, differing only in which branches targets builds (tarpolyglot_cross() for all combinations of several inputs, tarpolyglot_head() / tarpolyglot_tail() / tarpolyglot_slice() / tarpolyglot_sample() for a subset). The plain targets patterns (map(), cross(), and so on) keep working unchanged; the tarpolyglot_* helpers are simply the opt-in that adds compile-once on Rust.

They fall back to the plain pattern on Python and Julia

The same helpers are accepted by tar_target_py() and tar_target_jl(), where they are rewritten to the plain targets pattern: Python and Julia reuse a live interpreter, so there is nothing to compile and nothing to reuse. tarpolyglot_map() on a Python step is therefore exactly map(). This lets one pipeline branch every language with the same helper. The four files involved:

py/sq.py:

result = x ** 2

R/push.R (pre-script for the Python step, to_py hands x over):

to_py <- list(x = x)

rs/square.rs:

#[extendr]
fn square(x: f64) -> f64 { x * x }

R/post.R (post-script for the Rust step):

square(x)
list(
  tar_target(vals, c(10, 20, 30)),
  # Python: behaves exactly as map(vals).
  tar_target_py(
    name = py_b, script = "py/sq.py", inputs = c(x = "vals"),
    pre_script = "R/push.R", retrieve = "result",
    pattern = tarpolyglot_map(vals)
  ),
  # Rust: compiles once in py_b's sibling rs_b_rust_lib, reuses across branches.
  tar_target_rs(
    name = rs_b, script = "rs/square.rs", inputs = c(x = "vals"),
    post_script = "R/post.R",
    pattern = tarpolyglot_map(vals)
  )
)

The tarpolyglot_* helpers are recognised only inside the tarpolyglot constructors, which rewrite them before handing the pattern to targets. Used directly in a plain targets::tar_target() or targets::tar_target_raw() they raise an error (invalid dynamic branching pattern ... Illegal symbols found: tarpolyglot_map), because targets validates a pattern against its own fixed set of pattern functions and does not know these helpers. This cannot be corrected from tarpolyglot: targets looks its pattern functions up in a locked internal environment, so teaching it a new one would require modifying the targets package itself. In a plain targets target (which has no foreign code to compile, so nothing to gain) use the native map() / cross() / … directly.

Several Rust steps in one pipeline

A pipeline can contain as many tar_target_rs() steps as you like, mixing branching (tarpolyglot_map() and friends) and non-branching steps freely. Each step compiles its own library independently (or once, with a pattern helper), and the libraries do not interfere with one another, even when several run in the same crew worker. z reuses rs/scale.rs (content shown in “Object output” above); noise is a second, unrelated step in the same pipeline, rs/rng.rs (a small linear congruential generator, no extra crate needed):

#[extendr]
fn rng_normal(n: i32, seed: i32) -> Vec<f64> {
    let mut state = seed as u64;
    (0..n).map(|_| {
        state = state.wrapping_mul(6364136223846793005).wrapping_add(1);
        ((state >> 33) as f64) / (u32::MAX as f64)
    }).collect()
}

R/rng_post.R:

rng_normal(5L, seed)
list(
  tar_target(vals, c(10, 20, 30)),
  tar_target(seed, 42),
  # Branching, compile-once.
  tar_target_rs(z, script = "rs/scale.rs", inputs = c(petal = "vals"),
                post_script = "R/scale_post.R", pattern = tarpolyglot_map(vals)),
  # A second, non-branching Rust step in the same pipeline.
  tar_target_rs(noise, script = "rs/rng.rs", inputs = c(seed = "seed"),
                post_script = "R/rng_post.R")
)

Compile once, reuse in other steps

The library a Rust step compiles is an ordinary R object, so you can compile once in one target and reuse the functions in other, unrelated steps, with no recompilation. Two exported helpers make this explicit:

rs/math.rs:

#[extendr]
fn add_one(x: f64) -> f64 { x + 1.0 }

R/use_math.R: the compiled add_one() and the input x are in scope:

add_one(x)
list(
  tar_target(x, 5),
  # Compile once; the result is stored like any target value (rds by default, or qs).
  tar_target(mathlib, tarpolyglot::compile_rs_lib("rs/math.rs")),
  # Reuse it in a different step, with no recompilation.
  tar_target(y, tarpolyglot::run_rs_step_prebuilt(
    mathlib, post_script = "R/use_math.R", inputs = list(x = x)))
)

Because a script can define several #[extendr] functions, one compiled library can serve many steps. This is the recommended way to share Rust code across a pipeline: compile the functions once, then call them wherever you need them.

Using more than one library in a step

A single step can also use several compiled libraries. Call run_rs_step_prebuilt() once per library and combine the results. libA reuses rs/math.rs above; rs/lib_b.rs is a second, independent script:

#[extendr]
fn times_ten(x: f64) -> f64 { x * 10.0 }

R/a.R:

add_one(x)

R/b.R:

times_ten(x)
list(
  tar_target(x, 5),
  tar_target(libA, tarpolyglot::compile_rs_lib("rs/math.rs")),
  tar_target(libB, tarpolyglot::compile_rs_lib("rs/lib_b.rs")),

  tar_target(combined, {
    a <- tarpolyglot::run_rs_step_prebuilt(libA, post_script = "R/a.R", inputs = list(x = x))
    b <- tarpolyglot::run_rs_step_prebuilt(libB, post_script = "R/b.R", inputs = list(x = x))
    a + b
  })
)

The libraries are loaded one at a time (see “Current limitations”), so call them in sequence like this rather than trying to use functions from two libraries in the same expression.

Current limitations

Windows toolchain note

rextendr requires the Rust GNU toolchain on Windows (to match R’s mingw ABI). Either set it as the rustup default (shown above) or pass it per step:

tar_target_rs(
  name = petal_z,
  script = "rs/scale.rs",
  inputs = c(petal = "petal_length"),
  post_script = "R/scale_post.R",
  toolchain = "stable-x86_64-pc-windows-gnu"   # sets RUSTUP_TOOLCHAIN for the build
)

Trade-offs vs Python/Julia

Because a Rust step compiles rather than using a live interpreter:

See ?tar_target_rs and ?run_rs_step for the full argument reference, ?tarpolyglot_map, ?compile_rs_lib, and ?run_rs_step_prebuilt for compile-once and library reuse, and vignette("get_started") for how Rust steps fit alongside Python/Julia and crew.