h5_threads() sets how many threads
h5lite uses for parallel work, currently the Blosc and
Blosc2 codecs. The H5LITE_NTHREADS environment variable
sets the default. Threads need hdf5lib 2.2.0.1 or later.start). start and
count must now be whole numbers."bytes" encoding are now rejected before writing
starts.h5_move() no longer deletes a group moved into itself
(e.g. "g" to "g/sub"); this is now an
error.names,
dimnames, and row.names) are now kept
consistent: h5_move() moves a dataset’s scales with it,
h5_delete() and overwriting delete them with it (scales not
named after the dataset, such as netCDF coordinate variables, are
detached but kept), deleting a scale detaches it, and writing a scale
never overwrites an unrelated object (previously, writing x
replaced anything named x_names, and could rewrite the
names of a moved dataset).h5_write(list, file, name, attr = ...) no longer
replaces the dataset name with a group; lists can’t be
written as attributes.list(a = 1, 2), deleting
the group).POSIXct times are now stored in UTC (previously local
times were labelled Z), with fractional seconds.
POSIXlt is now written like POSIXct instead of
as an empty group.utf8[n] and fixed-length strings are no longer
truncated in the middle of a multibyte character, and their width is
measured in UTF-8 bytes.as = "ascii" now transliterates every string, not just
the first.70000 as float16, 2^63 as
int64) are now an error instead of becoming
Inf or wrapping around.blosc2_truncate no longer corrupts 32- and 64-bit
integers, including those in data frames (it zeroed their low bits: with
blosc2_truncate = 10, every int32 below 8192
read back as 0), or turns NA into NaN or
Inf. It now applies only to float32 and
float64 data without NA or
NaN.int64 NA values now read back as
NA instead of -9.22e18, and
uint64 values above the int64 range read with
as = "bit64" become NA (with a warning)
instead of being clamped.dim, class, …), no
longer produce malformed or reshaped results. Factors with empty-string
levels can no longer be written.h5_write(list, file, "/"),
h5$write(list)).h5_create_file() and h5_open() now work on
read-only files.as fixes: an unnamed as (or
non-@ key) no longer applies to attributes when writing;
as is now honored for factors (which can be stored as
strings, e.g. as = c(grade = "ascii[1]")), raw, complex,
integer64, NULL, and nested lists; "skip" and
"null" must be spelled out rather than abbreviated;
type-class keys like .int only match numeric data when
reading; "auto" keeps the automatic type; attributes read
or written directly accept plain or @ keys; and
"null" drops data frame columns when reading.start on a data frame is now an error,
and start/count on a group apply to each of
its datasets.h5_length() returns 0 for NULL datasets and
attributes.h5_ls() and h5_names() (for groups) now
return names in the order sort() puts them, instead of an
arbitrary order, and full.names = TRUE returns absolute
paths. h5_read() on a group and h5_str() now
follow the order objects were created (in files written by this version;
name order otherwise), so lists read back in the order they were
written.h5_str() shows names longer than 255 bytes, and
h5_str() and h5_attr_names() no longer show
HDF5’s DIMENSION_LIST/REFERENCE_LIST
attributes.h5_write() and h5_read() fail on Windows with
older versions of R (such as R 4.2), and as keys now match
an object named ~. Both came from R’s
basename(), which is meant for file paths.h5_open() handle methods now accept the same arguments
as the corresponding functions
(e.g. h5$read(name, start, count)), ".." works
in absolute paths, and a relative file path keeps working after
setwd().I() around a length-one POSIXct now writes
a scalar, and its warning for longer objects appears once.h5_inspect() reports filter parameters as non-negative
numbers."blosc1", "blosc2",
"blosc2-ndlz", "blosc1-snappy", standard
"blosc*-lz4") and "blosc2-zfp-acc-*" storing
data uncompressed."blosc1-gzip" and "blosc2-gzip" now use
Blosc instead of plain gzip.float_rounding no longer corrupt
NA, NaN, or Inf: ZFP falls back
to gzip and float_rounding is skipped for data containing
them.print(h5_inspect()) reads Blosc2’s settings in the new
layout as well as those written by earlier versions (including the
blosc2_truncate bits of 1-D datasets and of negative
settings) and by other software, lists the pre-filters in the order they
run (truncation, delta, shuffle), and shows when Blosc2’s ZFP codecs
declined a dataset.blosc2_delta and blosc2_truncate are no
longer passed to "blosc2-ndlz" and
"blosc2-zfp-...", which never ran them.
print(h5_inspect()) listed them anyway, and
blosc2_truncate = 8 made NDLZ use 8x8 cells.
print(h5_compression("blosc2-ndlz")) no longer claims a
bitshuffle.blosc2_truncate must be between -51 and 52, and not 0.
A setting that keeps no bits or drops a whole mantissa used to leave the
data uncompressed. For float32 data, settings that would
drop 23 bits or more are skipped."lz4-[level]" now passes the requested LZ4-HC level
instead of always using level 9."blosc2-zfp-acc-0",
which silently used a tolerance of 1) is now an error.float_rounding on float16 or
bfloat16 data no longer makes the write fail; HDF5’s
Scale-Offset filter can’t handle them, so it is skipped.compress = "none" no longer chunks or shuffles the
data, and compress = TRUE now selects gzip instead of no
compression."blosc2-zfp-*" now falls back to
the default shuffle + gzip pipeline."zfp-...") now falls back to the
default shuffle + gzip pipeline for data whose chunks have no dimension
longer than one (e.g. a single value) or more than four, which ZFP can’t
compress. The write used to fail.h5_inspect() no longer prints large chunk dimensions in
scientific notation.vignette('compression') now
covers every bundled codec, lossy compression, sharing files with other
software, and chunk tuning.These take effect when h5lite is built with hdf5lib 2.2.0.1 or later.
h5_threads().blosc2_truncate bits into the first. So
hdf5plugin describes them by their last pre-filter,
h5py can append to them, and h5py can read 1-D
Blosc2 datasets. hdf5plugin used to read the pre-filter
bitmask h5lite recorded as a single filter code, which isn’t a valid one
with blosc2_delta or blosc2_truncate. 1-D
Blosc2 datasets written earlier can’t be read by h5py until
they are rewritten."blosc2-zfp-..." now compresses arrays of more than
four dimensions, and chunks with a dimension shorter than 4, which it
used to store uncompressed. Data of fewer than four values is stored
uncompressed, and print(h5_inspect()) shows
e.g. blosc2 [zfp-acc] (declined: stored uncompressed).int_packing or float_rounding. It used to
refuse every chunk with two or more dimensions after Scale-Offset,
leaving them packed but otherwise uncompressed."snappy" datasets are written in the block layout of
snappy-cuda, the filter registered for Snappy’s HDF5 filter ID. h5lite
built with hdf5lib 2.2.0.0 or earlier can’t read them; older ones can
still be read.HDF5_PLUGIN_PATH. hdf5lib’s read-only BitRound
shim claimed SZ3’s ID, which made them unreadable; it now uses
BitRound’s own, 32032, which h5_inspect() names
bitround.‘%d’ directive output may be truncated in
utils.c.h5_compression() and
h5_inspect().vignette('compression').vignette('partial-reading').h5_length() and h5_dim() now return
numeric values instead of integer to support very large datasets.