F17.6 — Durable in-process job queue
| State | New |
| Parent | E17 |
| Tags | deferred |
| Predecessors | F17.4, F6.2 |
| Source | Owner request, 2026-10-21 |
| Entry criteria | a second application needs the queue |
Stories: 3 New
| Story | State | Depends on |
|---|---|---|
| S17.6.1 — Settle the shared surface with a second application | New | — |
| S17.6.2 — Build the models, worker and context | New | — |
| S17.6.3 — Build the SSE delta stream | New | — |
A single-user application that runs long work (a library scan, a bulk edit, a download) needs that work to survive a restart, report progress, be cancelled part-way and show its log, without a broker to install. Apollo wrote a SQLite-backed worker for this. This page is a design sketch to be settled when a second application needs one; PhotoTidy, which already has its own event broker, is the likely second consumer. Nothing here is built, and nothing starts until the entry criterion holds.
Context
Apollo does not wait for it; its worker stays in the application. One consumer shows what the code is, not what the shared surface should be, so the sketch records Apollo's shape and the second application revises it.
Async, unlike Apollo's worker. Apollo runs a poller thread and a thread pool
over synchronous sessions. The shared worker runs on the event loop over
F17.4's AsyncEngine, matching the async
package it lives in: a handler is a coroutine, does blocking file or subprocess
work through asyncio.to_thread, and reaches a single-owner library through
await actor.run(...).
Design
Where. A new rn_forge.sqlalchemy.jobs subpackage behind a jobs extra,
over F17.4's engine and migrations. The SSE
helper lives in rn_forge.fastapi.sse
(F6.2) and is the one piece in the FastAPI
package.
Models and migration. Job (id, type string, state, JSON payload, JSON
result, error, progress 0–100, progress note, cancel-requested flag, created,
started, finished) and JobLog (id, job id with cascade delete, timestamp,
level, message), over the package's Base. JobState is pending, running,
succeeded, failed, cancelled. type is a string so an application's
vocabulary needs no shared enumeration. The package ships an Alembic revision an
application includes in its own script location.
JobWorker:
type Handler = Callable[[JobContext], Awaitable[dict[str, Any] | None]]
class JobWorker:
def __init__(
self,
sessions: async_sessionmaker[AsyncSession],
handlers: Mapping[str, Handler],
*,
resources: Mapping[str, object] | None = None,
on_terminal: Callable[[str], None] | None = None,
max_workers: int = 2,
autostart: bool = True,
history_limit: int = 500,
idle_poll: float = 2.0,
shutdown_grace: float = 10.0,
) -> None: ...
async def start(self) -> None: ...
def notify(self) -> None: ...
def set_paused(self, paused: bool) -> None: ...
async def close(self) -> None: ...
start and close run from the application's lifespan. The poller is one
asyncio task, and at most max_workers handler tasks run at once.
- Bounded claim. The poller claims no more pending jobs than it has idle
handler slots, with a conditional
UPDATE … WHERE state = 'pending'. Claiming ahead of capacity would show a job as running when it sits in a queue. - Wake and poll.
notify()sets anasyncio.Eventthat wakes the poller on enqueue; the idle poll is the safety net for rows from another process. - Autostart and hold. With
autostart=Falsepending jobs are held, not replayed, untilset_paused(False); a count of held jobs is exposed. - Start-up recovery. Jobs left
runningby the previous process becomefailedwith an "interrupted by restart" error, and history beyondhistory_limitis pruned oldest first. - Shutdown.
close()stops claiming, asks running jobs to cancel, waits up toshutdown_grace, then cancels the remaining handler tasks and logs them. on_terminal(job_type)runs after every job, success or not.- Terminal recording. A handler's
JobCancelledrecordscancelled; an exception recordsfailedwith the traceback logged;asyncio.CancelledErrorrecordscancelledand re-raises. A result the JSON column cannot store recordsfailed, never a row stuck at 100% running.
JobContext: await progress(pct, note), await log(level, message)
(durable, shown to users), await cancelled() with the database read throttled
to once per cancel_poll seconds and sticky once seen, await checkpoint()
raising JobCancelled, payload, and resources, the mapping passed to the
worker.
The thread actor is an optional resource, not a constructor argument. The
worker knows nothing about F17.2. An application that
owns an actor passes it as resources={"library": actor}, and a handler reads
ctx.resources["library"]. Apollo's worker takes the actor as a required
argument because it has one; the shared version must not, or an application
without one inherits a concept it does not use. A handler awaits
ctx.resources["library"].run(fn).
SSE delta stream. job_events(sessions, *, interval, include_logs) is an
async iterator of ServerSentEvents for
F6.2's sse_response: a first snapshot,
then one frame per job whose state, progress or note changed. Apollo polls once
a second, writes its own frame function, sends a comment heartbeat every 30
ticks and wraps a StreamingResponse loop; sse_response and its ping replace
all three.
Prior art: Apollo's job worker, its Job and JobLog models and job states,
and its job router's delta and stream functions.
Tests (when built): a bounded claim never marks more than max_workers jobs
running; a handler's asyncio.to_thread work does not block another job's
progress write; a restart fails jobs left running; notify runs a job before
the idle poll; cancel is observed at checkpoint within cancel_poll; a
failing handler records failed and logs a traceback; an unserialisable result
records failed; held jobs do not run until resumed; history pruning keeps the
newest history_limit terminal jobs; the delta stream emits a snapshot then
only changes.
S17.6.1 — Settle the shared surface with a second application
State: New
Acceptance: a second application's requirements are written into this page, the sketch is revised where they differ from the first, and the open questions are answered or recorded as rejected options.
S17.6.2 — Build the models, worker and context
State: New
Acceptance: rn_forge.sqlalchemy.jobs behaves as under Design with the
listed tests passing, an Alembic revision applies to an empty SQLite database
through F17.4's run_migrations, and the
worker imports nothing from rn_forge.commons.runtime.actor.
S17.6.3 — Build the SSE delta stream
State: New
Acceptance: job_events feeds sse_response in a test application; a
client reads a snapshot and then only the changed job.
Acceptance
Written when S17.6.1 settles the design; each line is tagged with its story.
Considered and rejected
- A broker-backed queue (Celery, a message bus). Local-first applications cannot ask the user to install and supervise a second service.
- The actor as a constructor argument. See Design.
- A thread-pool worker over synchronous sessions, as Apollo has. The package is async only (F17.4), and an async worker leaves only the single-owner library on a dedicated thread.
Open questions
- Should the handler registry key on a string, as sketched, or on an application-supplied enumeration?
- Does PhotoTidy's event broker want the same persisted log, or only progress?