Skip to content

F22.12 — Tabular helpers without redundant copies

State Closed
Parent E22
Iteration Release 9
Start Date 2026-11-14
Closed Date 2026-11-15
Predecessors F22.3

Stories: 2 Closed

Story State Depends on
S22.12.1 — Preserve non-missing false values Closed (2026-11-14) —
S22.12.2 — Remove redundant row buffering Closed (2026-11-15) S22.3.2

Preserve legitimate values and remove avoidable row materialization.

Context

pandas.py string extraction uses a truthiness fallback after missing-value handling, which drops numeric zero and False. excel.py:393-396 materializes all dataset rows even though the workbook is write-only. The API still returns workbook bytes, so removing that list alone does not make the entire operation streaming.

S22.12.1 — Preserve non-missing false values

State: Closed (2026-11-14)

Separate absence from truthiness in string conversion. Keep pandas-native missing-value rules and required-field errors.

Acceptance:

  • Zero, False, empty string, None, NaN and pandas missing sentinels follow the documented distinct conversion behavior.
  • Optional pandas remains outside base imports and existing valid string/number/date cases remain covered.
  • A value pandas does not treat as missing becomes str(value).strip(). A missing value gives "" when optional and raises when required. "" is not missing. get_decimal_field raises on a False cell. The table is in the data usage guide.

Outcome: 0, 0.0 and False now read back as "0", "0.0" and "False" instead of "".

S22.12.2 — Remove redundant row buffering

State: Closed (2026-11-15) · Depends on: S22.3.2

Iterate the dataset directly where its contract permits. Review repeated frame/workbook conversion only where measurements show extra copies; keep output-format behavior.

Acceptance:

  • Large export shows a peak-memory comparison and no redundant complete row list in the write-only path.
  • Round-trip tests preserve headers, values, sheet name, number formats and unknown-column errors.
  • Documentation states the remaining in-memory dataset/output-bytes bounds and does not promise end-to-end streaming.

Outcome: write_xlsx iterates the dataset directly; for 50,000 × 10 cells the write-phase traced peak fell from 8.7 to 2.1 MB, and the round trip stays about 225 MB, dominated by the dataset, workbook and output bytes. ExcelAdapter.from_dataframe keeps its reparse, which is its editable-workbook contract.

Acceptance

  • S22.12.1: Zero, False, empty string, None, NaN and pandas missing sentinels follow the documented distinct conversion behavior.
  • S22.12.2: Large export shows a peak-memory comparison and no redundant complete row list in the write-only path.