read_dataset() and members() refuse an
empty format with a classed condition.
format = "" reached base::exists(""), which
throws an unclassed simpleError, so it escaped every
artoo_error_* handler a caller could have written.
write_spec() to Define-XML reports which files
informed the document when data is a folder, on every call
rather than only when something is unmatched. A define is a submission
artefact and the files behind it cannot be recovered from it
afterwards.
write_spec() to Define-XML warns when a file records
a different dataset than the name it was matched by, and when a derived
archive location names a file the folder does not hold. Matching is by
basename, so vs.json copied to dm.json used to
widen DM’s lengths from VS bytes in silence.
write_spec() to .xlsx documents what a
workbook cannot carry: a method with several formal expressions keeps
only the first, because a workbook gives each method one row with one
code cell. The write already named the methods it truncated; the help
page claimed the table survived whole.
read_spec() reads back a native JSON spec that names
no datasets. write_spec() wrote the file and
read_spec() refused it, saying to “pass at least a
dataset table” to a caller who had passed a path. A key
that is present but empty means the spec has none, which is not the same
as the key being absent.
write_spec() to Define-XML names every unreadable
file at once when data is a folder, rather than stopping at
the first. A folder holding three corrupt datasets took three runs to
discover. It still aborts rather than skipping them: a define written
from a silently reduced set of files is the wrong document produced
without complaint.
members() reads a gzipped dataset.
members("dm.ndjson.gz") aborted with
No codec handles the "gz" extension on a file
write_ndjson() produces and read_dataset()
reads. Gzip is peeled for the codecs that support it, so
dm.json.gz inventories as dm and
dm.parquet.gz stays unhandled, which is what
read_dataset() does with it too.
write_spec() to Define-XML accepts a folder for
data =, not only a named list of frames. Each dataset the
spec names is matched to a file whose basename is that name, ignoring
case, so a submission folder of dm.xpt and
ae.xpt no longer has to be retyped as
list(DM = dm, AE = ae). The folder is inventoried, not
descended. A dataset with no file is reported, not an error; a file the
spec does not name is left alone; a dataset matching two files aborts
rather than choosing, because two formats can disagree about byte width
and picking one silently would change the document.
write_spec() to Define-XML gains
data_format =, restricting a folder to the named formats.
It takes format names as artoo_formats() lists them. Adding
it means write_spec() no longer accepts dat =
as an abbreviation of data =, which R had been resolving by
partial matching.
members() gains format =, restricting
the inventory to the named formats. It takes format names as
artoo_formats() lists them, not file extensions, so
"parquet" claims both .parquet and
.pq; several names are a set rather than a precedence
order. The default NULL inventories every format, which is
what members() did before. The reason to pass it is a
directory holding one dataset in two formats, where the full inventory
reports both.
read_spec() refuses an
Analysis Criteria sheet missing Display,
Result or Dataset, naming the column. It used
to tolerate the absence silently: the join key matched nothing, the
whole sheet was discarded without a word, and the failure surfaced later
at write time naming the analysis results rather than the sheet that
caused it. Variables and Where Clause stay
optional, which is the split the format itself declares.
read_spec() links each dataset to the standard its
Standard cell names, minting a standards row
where none defines it — so a workbook-sourced define carries
def:StandardOID on every ItemGroupDef and the
rendered dataset headings show the standard, as CDISC’s own examples do.
A Standard column naming several standards no longer
aborts: each row keeps its own and @standard holds only the
primary. write_spec() to .xlsx accordingly
writes each dataset’s own standard rather than stamping the primary over
every row.
read_spec() reads an Analysis Criteria
sheet that omits its optional columns — one naming only datasets used to
die mid-read — and warns when that sheet overrides a result’s non-blank
Selection Criteria cell instead of discarding the cell
silently.
read_spec() keeps a sheet’s own
Description column when Label supplied the
label: it is unconsumed sponsor text, not a second spelling of one.
write_spec() to .xlsx likewise no longer
overwrites a foreign Description on the ValueLevel sheet
with the label.
write_spec() to Define-XML derives a dataset’s
archive location (LF.<DATASET> pointing at
<dataset>.xpt) when the spec states none, reusing a
document already carrying that id rather than minting a second leaf
beside it — two leaves with one xs:ID failed schema
validation on artoo’s own workbook round trip. A dataset flagged as
having no data derives none, the carve-out CDISC’s own examples
make.
write_spec() to Define-XML writes the partly-decoded
and duplicated codelist shapes real sponsor specifications carry,
instead of refusing them at the schema gate as an artoo defect: a term
without a decode in a decoded list gets an empty Decode
(never the string "NA"), a coded value listed twice
identically collapses to its first row with a warning, one defined two
ways is refused naming the codelist and values, and a codelist seating
two terms at one OrderNumber is written in source order
without the attribute. Identical repeated method and comment rows
collapse the same way; contradictory ones are refused by id.
write_spec() to .xlsx writes the one
workbook shape both generations of tooling imports. The study sheet is
named Study, a value-level row names its condition by ID
and a WhereClauses sheet defines it, an analysis result’s
datasets sit on an Analysis Criteria sheet, and a label is
carried under both Label and Description.
Measured against the importer that the open-source edition of the
reference tooling ships: it makes the study sheet its initialising sheet
and therefore required, so a workbook naming it Define was
refused before a row was read, and it treats the value-level cell as a
foreign key, so a condition written there as an expression cost every
value-level row.
read_spec() reads a value-level label from a
Description column, which is how the older workbook
generation spells it. Reading one used to give every value-level row no
label, and artoo’s own submission-grade notice then reported the
absence, on a workbook that had them.
read_spec() and write_spec() round-trip
a workbook artoo wrote. A workbook-sourced spec carries its value-level
link in a different column from a define-sourced one, and the writer
only understood the second, so its own output could not be read back. No
test covered it; one does now.
read_spec() on a define.xml keeps a variable’s
reference to an external dictionary. The referential scrub predates
dictionaries being modelled and stripped every such binding, so an
adverse-events define came back with AEDECOD naming no
terminology at all while the dictionary it named was written out beside
it.
check_spec() no longer aborts on a variable whose
terminology is an external dictionary. A dictionary enumerates nothing,
so there is no membership to check; checking anyway stopped the
conformance run on every adverse-events dataset.
write_spec() writes an external dictionary as the
ExternalCodeList it is, and read_spec() on a
define.xml reads one back. A variable naming MedDRA or WHODrug used to
write a CodeListRef pointing at nothing – schema-valid, and
rejected by artoo’s own reference check – while a document carrying one
lost it and every reference to it on the way in.
read_spec() on a workbook reads the
Analysis Criteria sheet, which is where the older workbook
generation keeps an analysis result’s datasets. artoo recognised the
sheet name and never read the sheet, so a result authored that way
reached the writer naming no analysis dataset and the write refused
it.
write_spec() to .xlsx no longer
re-homes a where-clause check that names a variable in another dataset.
Which dataset an unqualified name belongs to is decided by the row the
condition sits on, and the writer was deciding it from the condition’s
first check instead – so a VS value conditioned on
DM.COUNTRY came back conditioned on
VS.COUNTRY, selecting different records, with nothing
said.
read_spec() and write_spec() handle a
where-clause value containing the word and or
or. "Nausea and vomiting" is a real preferred
term, and the condition splitter was blind to quotes, so reading artoo’s
own workbook turned one check into two whose values were
"Nausea and vomiting".
write_spec() to .xml writes a page list
separated by spaces. A page cell written 4, 5 shipped its
comma into PageRefs, which is a space-separated
list.
write_spec() to .xlsx writes the study
sheet in the format’s own vocabulary. A spec read from a define.xml
carries the document’s identifiers, and none of them had a workbook
spelling, so the sheet went out reading define_version,
study_oid and metadata_version_oid – artoo’s
internal column names on a sheet a person reads and another tool
imports. It also states StandardName and
StandardVersion, which artoo read and never wrote back, so
a workbook it produced could not say which standard it
described.
read_spec() on a workbook resolves the standard its
Study sheet states. The sheet carries a name and a version as two
attributes; artoo kept both as unmodelled study fields, so
spec_standard() was NA and writing Define-XML
2.0 refused for want of the standard the sheet had named. A Study sheet
that names one now also yields the def:Standards row
Define-XML 2.1 asks for.
read_spec() on a workbook works out which documents
are the annotated CRF from their filenames, since the sheet has no
column for it. A document whose name ends in crf is the
annotated CRF, one with no filename belongs to no category, and an
explicit Role still wins.
write_spec() to .xml gives a method,
comment, codelist, document or standard named on a workbook sheet an OID
of its own kind, so a method called EXTRT is written as
MT.EXTRT. Names taken straight from the sheet collided
across kinds and the document failed its own schema. The spec keeps the
names the author wrote.
read_spec() on a workbook reads a method’s formal
expression. The Methods sheet carries one, in its
expression context and code columns, and artoo read those into the spec
and then used them nowhere: a method authored with a formal expression
produced a define.xml with none. Writing a workbook puts the expression
back in the same two columns.
read_spec() reads a value-level condition from
either carrier: an expression in the ValueLevel sheet’s
Where Clause cell, or a clause ID there with a
WhereClauses sheet defining it. An analysis result’s
datasets are likewise read from a Selection Criteria cell
or an Analysis Criteria sheet. Which one a workbook uses is
a property of the sheet it came from, not of the workbook, so both are
read wherever they appear.
read_spec() on a workbook no longer loses a
value-level row’s Assigned Value, Source,
Pages or Predecessor. All four are columns of
the sheet, and all four were mapped for Variables and not
for ValueLevel, so a value-level
Origin = Predecessor wrote an origin describing
nothing.
read_spec() on a workbook no longer keeps the quote
characters that delimit a where-clause value.
AVISIT EQ "Week 24" selected records whose AVISIT was the
eight characters "Week 24" rather than the seven.
read_spec() on a workbook reads a codelist’s
Comment as the comment reference it is, and a variable may
name an external dictionary in its Codelist column, which
is where a workbook puts it. Reading an adverse events spec aborted
asking for terms a dictionary does not have.
read_spec() and write_spec() keep an
analysis variable’s dataset qualifier straight, and a where-clause check
that leaves its own dataset now says so.
define_lint() is now lint_define(),
matching the verb-first grammar of every other export. The function is
new in this development version and has never been released, so nothing
that ran before breaks.
read_spec() with datasets = no longer
keeps the codelists, methods, comments and documents belonging to the
datasets it dropped. Scoping removes the referrers, so what was left
over was an orphan artoo had just created: a spec narrowed to two ADaM
datasets wrote a define.xml its own linter flagged 32 times. An orphan
in an unscoped spec is left exactly where the author put it.
read_spec() on a define.xml is about four times
faster and no longer grows super-linearly. Every XPath matches by
local-name(), but xml2 rebuilt a namespace
prefix table from the whole document on each one.
read_spec() on a define.xml reads a value-level
ItemRef/@Role, and reports the codelist aliases,
CodeList descriptions and declared languages it has no
column for, rather than dropping them in silence.
write_spec() to .xml no longer aborts
when data = covers only some of the datasets sharing an
ItemDef OID; the widest measurement applies to the whole
group.
write_spec() to .xml no longer reports
an ADaM spec as missing datasets$domain, which is an SDTM
concept ADaM leaves blank.
write_spec() to .xml says what a bare
data frame is when data = gets one. A data frame is a named
list, so it reached the per-element check and was refused for a column
not being a data frame, naming the column.
The bundled sdtm-spec.xlsx and
adam-spec.xlsx are rebuilt. They were written before the
where-clause fix below, so reading the package’s own example workbook
warned that 23 value-level rows named a clause it did not
define.
write_spec() on a Pinnacle 21 workbook now names the
individual columns a workbook cannot carry, not just the one slot it has
no sheet for. The sheets added for Define-XML closed the slot-level gaps
and opened column-level ones: Standards has no column for
which standard is primary, and none of the sheets has one for an
ItemOID.
write_spec() on a Pinnacle 21 workbook keys a
ValueLevel row to the WhereClauses sheet by ID. It wrote
the rendered expression there, which left every value-level row of an
SDTM define naming a clause the same workbook did not define.
write_spec() renders HTML through the stylesheet the
document’s processing instruction actually names, so
stylesheet = "acme.xsl" and html = TRUE no
longer render through the bundled sheet and disagree with the
browser.
write_spec() writes Define-XML when given a
.xml path, so a Pinnacle 21 workbook or a native JSON spec
becomes a submission-grade define.xml in one call. The document is
schema-validated before it reaches its destination, so an invalid one
never overwrites a good file. Value-level metadata is emitted whole: the
parent variable’s def:ValueListRef, the
def:ValueListDef, an ItemDef per value-level
row, and the def:WhereClauseRef and
def:WhereClauseDef behind it. External dictionaries are the
one thing not written yet; a populated dictionaries table
is reported rather than emitted. Needs the xml2
package.
read_spec() and write_spec() no longer
lose a multi-value IN where clause through a Pinnacle 21
workbook. The workbook holds one row per range check and the spec holds
one per value, and writing them row for row made
PARAMCD IN (A, B, C) read back as three ANDed one-value
checks, which select nothing.
read_spec() on a Pinnacle 21 workbook reads an
Origin Document column on the Variables sheet, so a page
reference keeps what it points at. read_spec() on a
Define-XML document now records which dataset defines a where clause’s
target, which a workbook has a column for and an OID does not.
Reading a spec scoped to some datasets no longer leaves its where clauses and analysis results referencing the others.
write_spec() now emits a variable’s predecessor and
assigned value. Define-XML has no attribute for either, so both go in
the origin’s description, which is where the CDISC stylesheet renders
them; artoo read them and wrote an empty origin, so an ADaM define lost
its traceability.
write_spec() now emits CRF page references from a
workbook. A page number with no document defaults to the annotated CRF
when the spec names one, and is refused rather than dropped when it does
not. read_spec() on a Pinnacle 21 workbook reads a
Role column on the Documents sheet, so an author can
designate which document is the annotated CRF.
write_spec() refuses a value-level row with no where
clause. Written out it was an ItemRef with no children:
schema-valid, nothing dangling, and asserting that the definition
applies to every row of its parent.
lint_define() gains
define_unconditional_value and
define_no_data_uncommented. Both catch things the schema
cannot see (a well-formed element, an optional attribute) and the
reference checks cannot see (nothing to dangle).
write_spec() on a .xlsx path now writes
the WhereClauses, Dictionaries,
Standards and analysis-results sheets.
read_spec() learned them when the Define-XML work landed,
so a workbook round trip was losing exactly what artoo had just taught
itself to read.
write_spec() gains data, which lets the
datasets a define describes inform what it says. A blank
length is filled from the real maximum byte width and a
stated one shorter than the data is widened, because a length below the
real maximum is a conformance finding; a length longer than the data is
left alone, because it is a claim about the domain rather than about one
extract. Value-level metadata is derived for the standard findings
shapes (a result keyed by its test code, TSVAL by
TSPARMCD, QVAL by QNAM,
AVAL and AVALC by PARAMCD), each
derived row carrying the type and width of the rows it covers, and a
variable the spec already describes is never touched. A dataset with no
records is flagged def:HasNoData when it carries a comment
explaining the absence.
write_template() writes a blank Pinnacle 21 workbook
with the sheets and headers read_spec() recognises, so a
specification can be authored from the shape the reader wants rather
than guessed at. Every header comes from the reader’s own maps, so the
template cannot offer a column that would be silently ignored. A
version = "2.0" template omits the four columns Define-XML
2.1 introduced.
write_spec() gains html, which renders
the define.xml through its own CDISC stylesheet into real HTML. Browsers
are dropping XSLT support, so a document that renders only by being
opened in one is on its way to being unreadable. Needs the
xslt and callr packages; the transform runs in
a separate process because libxslt and libxml2’s schema validator
corrupt each other’s state in one session.
write_spec() no longer overwrites a stylesheet
already sitting beside the output, so a customised rendering survives a
rewrite.
write_spec() warns when the define.xml it wrote is
valid but not submission-grade, naming every column nothing fills. None
of them is required by the schema, and every one draws a conformance
finding, so a spec that is incomplete partway through a study is still
written rather than refused.
read_spec() on a Pinnacle 21 workbook now
forward-fills the merged cells on the analysis-results sheets, as it
already did for every other sheet. A workbook merges the display cell
across a display’s results and the result id across a result’s analysis
datasets; without the fill those rows were dropped.
A where clause may now qualify a variable in a different dataset,
which is what a value-level condition on DM.COUNTRY means.
A variable two datasets define is refused rather than resolved to one of
them.
read_spec() on a Define-XML 2.0 document now reads
its standard as a standards row rather than one
concatenated string. A version containing a space
(3.1.2 Amendment 1 is a real one) used to have half of it
moved into the standard’s name on the way back out, and a document
naming no standard at all used to be written back asserting the standard
was NA.
read_spec() on a Define-XML document now reports
every place it reads one of several: a second
def:DocumentRef, def:PDFPageRef, or
TranslatedText. All four bundled CDISC examples lose a
def:DocumentRef this way, and the loss was invisible
because the writer dropped it too.
read_spec() and write_spec() now carry
the document’s build provenance (Originator,
SourceSystem, SourceSystemVersion) and the
MetaDataVersion’s own comment.
Analysis Results Metadata (ARM v1.0) is read and written, in both
Define-XML versions. read_spec() fills
arm_displays and arm_results from
arm:AnalysisResultDisplays, and write_spec()
emits them back. One analysis result over several analysis datasets
keeps each dataset’s own where clause and analysis variables, which is
why arm_results is one row per result and dataset. Before
this, reading an ADaM define and writing it back orphaned every leaf,
where clause and comment that only its analysis results
referenced.
read_spec() on a Define-XML document now reads
def:PDFPageRef/@Title, which names the table or listing a
page holds. Every analysis-results page reference in the CDISC ADaM
example carries one. It is 2.1-only, and is withheld when writing
2.0.
write_spec() writes Define-XML 2.0 as well as 2.1,
and converts between them. The version follows the spec’s own
define_version unless version says otherwise.
The switch is exactly what the two standards renamed:
def:Standards against
def:StandardName/def:StandardVersion,
def:Class as an element against an attribute,
def:Context and def:Origin/@Source in 2.1
only, and the origin vocabulary (2.0’s CRF and
eDT are 2.1’s Collected). A downgrade warns
once, naming everything 2.0 cannot carry; a 2.1 origin with no 2.0
spelling is refused rather than approximated.
read_spec() on a Define-XML document now reads a PDF
page RANGE. A def:PDFPageRef states its pages as either a
PageRefs list or a
FirstPage/LastPage range, and only the list
was read, so every range was dropped silently. Both now land in
pages, a range written as "4-5".
write_spec() gains ..., which the
Define-XML path reads for version, created,
stylesheet, and validate. Freeze
created for a byte-reproducible submission build.
read_spec() on a Define-XML document now keeps the
document’s own identity: Study/@OID, FileOID,
the MetaDataVersion’s OID, name and description, and the ODM
@Context. write_spec() writes them back, so a
document read and written keeps every identifier an external reference
may already name, rather than having fresh ones minted from the study
name.
read_spec() on a Define-XML document now reports
what it drops. An external codelist (MedDRA, WHODrug, ISO
3166) and every reference to it are still dropped, and an
ItemDef carrying more than one def:Origin
still keeps only the first, but both now warn: a loss that reader and
writer share is invisible to a round trip, and one nothing reports is
worse than one that fails. A value-level item selected by more than one
def:WhereClauseRef is now refused outright, because those
are combined with OR and keeping the first silently narrows which rows
the definition applies to.
read_spec() on a Define-XML document now carries a
value-level row’s whole ItemDef: its origin, comment,
display format, significant digits, and SAS field name, alongside the
def:WhereClauseRef it names. Previously only the label,
type, length and codelist survived, so a document read and written back
lost every value-level origin.
read_spec() on a Pinnacle 21 workbook now reads the
WhereClauses, Dictionaries,
Standards, and analysis-results sheets. A workbook with no
WhereClauses sheet has its value-level conditions parsed
from the free-text column instead, so both workbook generations are
read.
Where-clause values are now parsed correctly for every comparator. A value list is split only outside quotes, so a value containing a comma survives, and parentheses are stripped from scalar comparisons as well as list ones. An unknown comparator, or a disjunction that Define-XML cannot express, is refused with an explanation rather than silently reinterpreted.
artoo_spec() gains six tables:
standards, where_clauses,
method_expressions, arm_displays,
arm_results, and a reserved dictionaries.
read_spec() on a Define-XML document now fills the first
three, so the def:Standards block, the structured
RangeChecks behind every value-level condition, and each
method’s formal expressions survive a read instead of being
discarded.
A specification saved by an earlier version of artoo is upgraded
automatically the first time any artoo function touches it, and says so
once. Previously such an object passed every type check and then failed
on the first new field. Save it again with write_spec() to
make the upgrade permanent.
The native spec JSON format is now version 2, carrying the six new tables. Reading a version 1 file still works.
read_spec() on a Define-XML document now reads the
metadata it previously discarded: SASFieldName,
def:Origin/@Source, the Origin document and page reference,
NCI controlled-terminology codes at both codelist and term level,
codelist
Name/DataType/SASFormatName, and
the ItemGroupDef attributes a submission needs
(Domain, SASDatasetName, Purpose,
Repeating, IsReferenceData,
ArchiveLocationID). Each document leaf now records which
container owns it.
read_spec() no longer loses class on
Define-XML 2.0 documents. def:Class is a child element in
2.1 but an attribute in 2.0, and only the element was read, so every 2.0
document came back with an empty class column.
lint_define() reports the reference-integrity
defects XML Schema cannot express: an OID reference that resolves to
nothing, and a definition nothing references. Findings are directional,
and an orphaned def:ValueListDef is an error rather than a
warning because every value-level definition it holds then renders
nowhere. Codelists backed by an external dictionary, and those reached
through RoleCodeListOID, are exempt from the orphan
check.
validate_define() schema-validates a Define-XML 2.0
or 2.1 document against the CDISC schemas, which artoo now bundles, so
validation runs offline with no network round trip. The version is
detected from the document’s namespace unless you assert one. Note that
schema validity is a floor: a reference that points at nothing, and a
definition nothing points at, are both invisible to XML Schema.
write_json() and write_ndjson() now
emit columns that conform to the Dataset-JSON v1.1 schema’s closed
Column vocabulary: origin, codelist, and
significantDigits moved out of the columns
array into the namespaced _artoo extension block (joining
informats), and label is always emitted, as
"" when a variable or dataset has none, instead of being
omitted. strict = TRUE output now validates against the
official CDISC schema, and third-party Dataset-JSON readers that require
the per-column label key can read artoo’s files. Reads map
an empty label back to absent, so round-trips remain lossless.
artoo_checks() gained an
invalid_encoding dimension (on by default):
check_spec() flags character values whose bytes are not
valid UTF-8, the signature of a source read under a mis-declared
encoding, before a writer aborts on them.
artoo_encodings() name resolution now accepts the
SAS OEM/DOS encoding names (pcoem437,
pcoem850, pcoem852, pcoem858,
pcoem862, pcoem866, msdos737),
and the reference table lists the PCOEM437 /
PCOEM850 rows.
write_xpt(), write_json(),
write_ndjson(), and write_parquet() accept
on_invalid = "translit", folding smart punctuation (curly
quotes, en/em dashes, ellipsis, bullet) to its exact ASCII form per the
SAS NLS punctuation table; characters with no fold still abort
loudly.
write_xpt(), write_json(),
write_ndjson(), and write_parquet() also
accept on_invalid = "fold": the punctuation fold plus the
ICU Latin-ASCII accent strip (Ö to O,
ß to ss, Æ to AE),
pinned as data so the result is identical on every platform; characters
neither table maps (the Euro sign) still abort.
write_xpt() now warns
(artoo_warning_encoding) when a value forces a column wider
than its spec-declared length, instead of widening silently; data is
still never truncated.
write_xpt() and the other writers’
on_invalid = "replace" now substitutes one ?
per unrepresentable character instead of one per byte (a curly quote
previously became ???).
New article: Migrating clinical data from WLATIN1 to UTF-8, including the smart-punctuation fold table and the byte-length migration recipe.
capabilities("long.double") so it skips on noLD builds,
where bit-exact double-to-string round-trips are not guaranteed by the
platform C library.Initial CRAN release. artoo is a lightweight, lossless, CDISC-native
reader and writer for clinical-trial datasets, built around one
canonical metadata model (artoo_meta) so that conversion
between any two supported formats is lossless by construction. Pure R
and lightweight, with no external SAS or Java runtime.
Reads and writes SAS XPORT (v5 and v8), CDISC Dataset-JSON v1.1,
NDJSON, Apache Parquet, and RDS. Every codec carries the full
artoo_meta — labels, CDISC data types, lengths, SAS display
formats, controlled-terminology references, and sort keys — so
any-to-any conversion preserves the complete metadata. For Parquet the
metadata rides as a metadata_json sidecar; a file written
by another tool with no sidecar degrades gracefully to a bare frame
rather than an error.
Generic read_dataset() /
write_dataset() dispatch on the file extension, with
read_xpt() / write_xpt() and the matching
read_json(), read_ndjson(),
read_parquet(), and read_rds() pairs as direct
entry points. Cross-cutting encoding, checks,
and created arguments flow through
....
Partial reads (col_select, n_max) on
every reader; gzip-transparent JSON and NDJSON; multi-member SAS XPORT
libraries via xpt_members() plus
read_xpt(member = ).
Numeric fidelity is exact end to end: a decimal
value is exchanged as a string at IEEE round-trip precision,
integer values beyond R’s 32-bit range stay numeric rather
than overflowing, and NaN / infinite values are rejected as
invalid CDISC numerics. Rows are sorted in C-locale (byte) order, so a
written file is deterministic across locales and matches SAS
PROC SORT for ASCII keys.
Encodings follow the IANA and SAS standards: the readers and
writers accept a charset name in either the SAS or R spelling (see
artoo_encodings()), character columns are transcoded to
UTF-8 and NFC-normalized on read, and the
on_invalid = c("error", "replace", "ignore") policy governs
invalid bytes uniformly across every writer.
artoo_spec() builds the canonical metadata model
from a Pinnacle 21 Excel workbook, a Define-XML 2.0 / 2.1 file, or a
native artoo JSON spec. read_spec() /
write_spec() dispatch on the file extension:
.xlsx writes a Pinnacle 21 workbook (Define-XML to P21 is
one composition), and the native JSON form is the lossless interchange
that round-trips a spec identically.
The spec is single-standard by construction:
@standard is resolved once from the explicit argument or
the source, and study-level fields are canonicalized to the CDISC ODM
vocabulary. Accessors include spec_standard(),
spec_variables(), spec_codelists(),
spec_methods(), and spec_comments().
set_type() returns a spec with one or more variables
retyped through the CDISC vocabulary; repair_spec() retypes
every variable a check_spec() run flags as fractional or
out-of-range under an integer data type, so a frame the
original spec would refuse coerces after one call.
apply_spec(x, spec, dataset, conformance = , na_position = )
coerces each column to its CDISC data type, orders the columns and sorts
the rows by the spec’s keys, and stamps the artoo_meta.
extra = c("keep", "drop") controls whether undeclared
columns survive; on_coercion_loss = c("error", "keep")
governs a coercion that would lose data. The pipeline never silently
fabricates or drops a column: an undeclared column is reported and kept,
a declared-but-absent column is reported and left absent.
check_spec() validates a data frame against its spec
across conformance dimensions toggled by artoo_checks();
check_study() runs it over a whole study and returns one
stacked findings frame; conformance() reads the findings
back off a stamped frame. validate_spec() checks a spec for
internal consistency against a bundled rule catalog, with no external
dependency.
decode_column() translates coded values to or from
their codelist decodes; sync_meta() reconciles a stamped
frame’s metadata after manual edits.
members() is the format-neutral inventory of the
dataset(s) a path holds, one row per dataset, dispatched through the
codec registry. columns() is the SAS PROC CONTENTS /
Universal Viewer variable pane over a stamped frame, a plain data frame,
or a file path. get_meta() / set_meta() read
and attach the artoo_meta.artoo_<severity>_<kind>,
artoo_<severity>, artoo_condition — so a
handler can catch a specific kind, a whole severity, or every artoo
condition. The data-protection conditions attach their evidence as data
(cnd$variables, cnd$findings) for programmatic
inspection.adam_spec (ADaMIG 1.1) and
sdtm_spec (SDTMIG 3.1.2), built reproducibly from the
official CDISC Define-XML 2.1 release examples and shipped also as
Pinnacle 21 workbooks under inst/extdata/. Demo datasets
come from the PHUSE Test Data Factory; the constructor tables
cdisc_adam_datasets / cdisc_adam_variables,
cdisc_sdtm_datasets / cdisc_sdtm_variables,
and the shared cdisc_codelists build a spec by hand. Every
bundled dataset conforms to its bundled spec, gated at build and test
time.vignette("artoo") plus task-oriented
web articles (specifications; conform and validate; formats and lossless
conversion; recipes), and a pkgdown reference site.