F18.3 — Fast, complete document reads and writes
| State | Closed |
| Parent | E18 |
| Iteration | Release 6 |
| Start Date | 2026-10-25 |
| Closed Date | 2026-10-25 |
| Predecessors | — |
| Source | Owner request, 2026-10-24 |
Stories: 4 Closed
| Story | State | Depends on |
|---|---|---|
| S18.3.1 — YAML through libyaml, with the pure emitter | Closed | — |
S18.3.2 — TOML through tomllib, package resources, brace-safe errors |
Closed | — |
| S18.3.3 — TOML header and array style | Closed | — |
S18.3.4 — as_mapping, as_list, as_strings |
Closed | — |
DocumentUtils and YamlUtils become the fastest way to read a config
document, not a slower convenience. They also cover the shapes a tool writes:
- YAML loads through libyaml.
- Plain TOML reads use
tomllib. - A document can be read from a package resource.
- A TOML document can be written with a comment header and one array element per line.
DictUtils and ListUtils gain typed reads of a bare value. Every
DocumentError built from parser text stops crashing on braces.
Context
Measured 2026-10-24 on Python 3.14 over a real repository's documents, the pure
ruamel YAML load took 1.69 ms per file against 0.35 ms with ruamel.yaml.clib,
and tomlkit plus unwrap took 1.07 ms against 0.15 ms for tomllib. kiln
imported PyYAML's C loader and tomllib directly for these reasons, and its
test suite reads the same Taskfiles and workflows hundreds of times.
ruamel.yaml 0.19.1 uses its C parser only with ruamel.yaml.clib, which the
oldlibyaml extra installs; the libyaml extra installs ruamel.yaml.clibz,
which 0.19.1 never imports. ruamel's C emitter must not be used: it folds long
lines differently from the pure emitter and raises OverflowError for
width=sys.maxsize, so loading in C with the pure emitter keeps the output
byte-identical. AppException formats its message with str.format(*args), so
a DocumentError built from parser text that contains a brace raised
KeyError.
Design
The feature shipped as designed in rn-forge-commons; the documents section of
the commons data guide (docs/guides/data.md) and the API reference document
the result.
S18.3.1 — YAML through libyaml. The base requirement is
ruamel.yaml[oldlibyaml], and _safe_yaml swaps in the pure emitter.
Round-trip mode stays pure Python by design.
S18.3.2 — TOML reads and package resources. DocumentUtils.loads reads
plain TOML with tomllib; tomlkit remains for read_document,
write_document, update and dumps. DocumentUtils.read accepts a
Traversable and names the source in a parse error, and every DocumentError
passes its variable parts as arguments, never pre-formatted.
S18.3.3 — TOML output style. dumps and write take header and
multiline_arrays, and DocumentUtils.multiline_arrays formats a round-trip
document's arrays with { key = value } inline-table spacing.
S18.3.4 — Typed reads of a bare value. DictUtils.as_mapping,
ListUtils.as_list and ListUtils.as_strings back the existing get_mapping,
get_list and get_strings, whose contracts are unchanged.
S18.3.1 — YAML through libyaml, with the pure emitter
State: Closed
Acceptance: as under Design; the four YAML tests pass, and every existing
YamlUtils, DocumentUtils and Config test passes unchanged.
S18.3.2 — TOML through tomllib, package resources, brace-safe errors
State: Closed
Acceptance: as under Design; the TOML, resource and brace tests pass.
S18.3.3 — TOML header and array style
State: Closed
Acceptance: as under Design; the header and multiline_arrays tests pass.
S18.3.4 — as_mapping, as_list, as_strings
State: Closed
Acceptance: as under Design; the new collection tests pass, and the
existing get_mapping, get_list and get_strings tests pass unchanged.
Acceptance
set -euo pipefail
cd packages/rn-forge-commons
uv run pytest -q tests/fs/test_documents.py -k 'libyaml or emitter or unfolded or brace' # S18.3.1 S18.3.2
uv run pytest -q tests/fs/test_documents.py -k 'toml or resource' # S18.3.2
uv run pytest -q tests/fs/test_documents.py -k 'header or multiline' # S18.3.3
uv run pytest -q tests/lang/test_collections.py # S18.3.4
uv run pytest -q # S18.3.1–S18.3.4
uv run python -c 'import _ruamel_yaml' # S18.3.1
uv run pyright && uv run --group docs mkdocs build --strict # S18.3.1–S18.3.4
Considered and rejected
- PyYAML's
CSafeLoaderbehindYamlUtils. It is about 2.5 times faster again, but it is a second YAML library beside ruamel, with YAML 1.1 rules (onloads asTrue). ruamel with libyaml is within 0.25 ms per file. - Reusing one
YAMLinstance per process. It saves about 0.07 ms per load, and aYAMLobject is not safe to share between threads. tomllibfor round-trip reads. It discards comments and formatting, which is what the round-trip methods exist to keep.multiline_arrayson by default. It changes the output of every existingdumpscaller.- The
libyamlextra (ruamel.yaml.clibz). ruamel.yaml 0.19.1 never imports it, so it installs a compiled module and still parses in pure Python. Revisit when a ruamel release uses it.