Skip to content

F18.3 — Fast, complete document reads and writes

State Closed
Parent E18
Iteration Release 6
Start Date 2026-10-25
Closed Date 2026-10-25
Predecessors —
Source Owner request, 2026-10-24

Stories: 4 Closed

Story State Depends on
S18.3.1 — YAML through libyaml, with the pure emitter Closed —
S18.3.2 — TOML through tomllib, package resources, brace-safe errors Closed —
S18.3.3 — TOML header and array style Closed —
S18.3.4 — as_mapping, as_list, as_strings Closed —

DocumentUtils and YamlUtils become the fastest way to read a config document, not a slower convenience. They also cover the shapes a tool writes:

  • YAML loads through libyaml.
  • Plain TOML reads use tomllib.
  • A document can be read from a package resource.
  • A TOML document can be written with a comment header and one array element per line.

DictUtils and ListUtils gain typed reads of a bare value. Every DocumentError built from parser text stops crashing on braces.

Context

Measured 2026-10-24 on Python 3.14 over a real repository's documents, the pure ruamel YAML load took 1.69 ms per file against 0.35 ms with ruamel.yaml.clib, and tomlkit plus unwrap took 1.07 ms against 0.15 ms for tomllib. kiln imported PyYAML's C loader and tomllib directly for these reasons, and its test suite reads the same Taskfiles and workflows hundreds of times.

ruamel.yaml 0.19.1 uses its C parser only with ruamel.yaml.clib, which the oldlibyaml extra installs; the libyaml extra installs ruamel.yaml.clibz, which 0.19.1 never imports. ruamel's C emitter must not be used: it folds long lines differently from the pure emitter and raises OverflowError for width=sys.maxsize, so loading in C with the pure emitter keeps the output byte-identical. AppException formats its message with str.format(*args), so a DocumentError built from parser text that contains a brace raised KeyError.

Design

The feature shipped as designed in rn-forge-commons; the documents section of the commons data guide (docs/guides/data.md) and the API reference document the result.

S18.3.1 — YAML through libyaml. The base requirement is ruamel.yaml[oldlibyaml], and _safe_yaml swaps in the pure emitter. Round-trip mode stays pure Python by design.

S18.3.2 — TOML reads and package resources. DocumentUtils.loads reads plain TOML with tomllib; tomlkit remains for read_document, write_document, update and dumps. DocumentUtils.read accepts a Traversable and names the source in a parse error, and every DocumentError passes its variable parts as arguments, never pre-formatted.

S18.3.3 — TOML output style. dumps and write take header and multiline_arrays, and DocumentUtils.multiline_arrays formats a round-trip document's arrays with { key = value } inline-table spacing.

S18.3.4 — Typed reads of a bare value. DictUtils.as_mapping, ListUtils.as_list and ListUtils.as_strings back the existing get_mapping, get_list and get_strings, whose contracts are unchanged.

S18.3.1 — YAML through libyaml, with the pure emitter

State: Closed

Acceptance: as under Design; the four YAML tests pass, and every existing YamlUtils, DocumentUtils and Config test passes unchanged.

S18.3.2 — TOML through tomllib, package resources, brace-safe errors

State: Closed

Acceptance: as under Design; the TOML, resource and brace tests pass.

S18.3.3 — TOML header and array style

State: Closed

Acceptance: as under Design; the header and multiline_arrays tests pass.

S18.3.4 — as_mapping, as_list, as_strings

State: Closed

Acceptance: as under Design; the new collection tests pass, and the existing get_mapping, get_list and get_strings tests pass unchanged.

Acceptance

set -euo pipefail
cd packages/rn-forge-commons
uv run pytest -q tests/fs/test_documents.py -k 'libyaml or emitter or unfolded or brace'   # S18.3.1 S18.3.2
uv run pytest -q tests/fs/test_documents.py -k 'toml or resource'                         # S18.3.2
uv run pytest -q tests/fs/test_documents.py -k 'header or multiline'                      # S18.3.3
uv run pytest -q tests/lang/test_collections.py                                           # S18.3.4
uv run pytest -q                                                                          # S18.3.1–S18.3.4
uv run python -c 'import _ruamel_yaml'                                                    # S18.3.1
uv run pyright && uv run --group docs mkdocs build --strict                               # S18.3.1–S18.3.4

Considered and rejected

  • PyYAML's CSafeLoader behind YamlUtils. It is about 2.5 times faster again, but it is a second YAML library beside ruamel, with YAML 1.1 rules (on loads as True). ruamel with libyaml is within 0.25 ms per file.
  • Reusing one YAML instance per process. It saves about 0.07 ms per load, and a YAML object is not safe to share between threads.
  • tomllib for round-trip reads. It discards comments and formatting, which is what the round-trip methods exist to keep.
  • multiline_arrays on by default. It changes the output of every existing dumps caller.
  • The libyaml extra (ruamel.yaml.clibz). ruamel.yaml 0.19.1 never imports it, so it installs a compiled module and still parses in pure Python. Revisit when a ruamel release uses it.