Skip to content

Tasks

rn_forge.commons.runtime.tasks

Parallel task execution via a managed thread pool.

Provides: Task: Abstract base class for units of concurrent work. Subclass and implement :meth:Task.run. TaskPool: Thread pool that submits :class:Task instances, tracks their futures, and reports failures on shutdown. Its :meth:TaskPool.map_settled fans a function over items. Succeeded, Failed, Settled: Per-item outcome of :meth:TaskPool.map_settled.

Typical usage::

from rn_forge.commons.runtime.tasks import Task, TaskPool

class FetchTask(Task):
    def run(self) -> None:
        url = self.get_prop("url")
        ...  # perform work

with TaskPool(max_workers=4) as pool:
    pool.submit(
        FetchTask("fetch-a", url="https://example.com/a"),
        FetchTask("fetch-b", url="https://example.com/b"),
    )
# all tasks have completed (or AppException is raised on failure)

Settled = Succeeded[T] | Failed

Outcome of one item: :class:Succeeded or :class:Failed.

Failed(error) dataclass

Outcome of an item whose function raised an :class:Exception.

Attributes:

Name Type Description
error Exception

The exception the function raised.

Succeeded(value) dataclass

Outcome of an item whose function returned.

Attributes:

Name Type Description
value T

The value the function returned; None is a valid value.

Task(name, **props)

Bases: ABC

Abstract base for units of work executed inside a :class:TaskPool.

Subclass and implement :meth:run. Pass keyword arguments as props; retrieve them later with :meth:get_prop.

Example::

class DownloadTask(Task):
    def run(self):
        url = self.get_prop("url")
        ...

pool = TaskPool()
pool.submit(DownloadTask("fetch-data", url="https://example.com"))
pool.shutdown()

Initialize a :class:Task.

Parameters:

Name Type Description Default
name str

Unique human-readable identifier for this task. Used as the thread name while the task is executing and as the key in :attr:TaskPool.tasks.

required
**props Any

Arbitrary keyword arguments stored as task properties and retrievable via :meth:get_prop.

{}

error property

The exception raised during :meth:run, if any.

Returns:

Type Description
Exception | None

The caught exception, or None when the task succeeded or has

Exception | None

not yet been run.

future property

The :class:~concurrent.futures.Future for this task's execution.

Returns:

Type Description
Future[Any] | None

The Future assigned by :meth:TaskPool.submit, or None

Future[Any] | None

before the task has been submitted.

name property

Unique identifier for this task.

Returns:

Type Description
str

The name string supplied at construction time.

props property

Dictionary of properties supplied at construction time.

Returns:

Type Description
dict[str, Any]

A mutable dict of all keyword arguments passed to __init__.

get_prop(key, default=None)

Return the prop value for key, or default if absent.

Parameters:

Name Type Description Default
key str

The property name to look up.

required
default Any

Value returned when key is not present. Defaults to None.

None

Returns:

Type Description
Any

The stored property value, or default.

run() abstractmethod

Execute the task's work.

Subclasses must implement this method. It is called by the thread pool via :meth:_start; do not call it directly. Any exception raised here is captured in :attr:error and re-raised so the :class:~concurrent.futures.Future records it.

TaskPool(max_workers=10)

Thread pool that submits :class:Task instances and tracks their results.

The type parameter _TaskType bounds the pool to a specific :class:Task subclass, enabling type-safe access to task-specific properties after completion.

Can be used as a context manager — :meth:__exit__ calls executor.shutdown(wait=True) so all tasks finish before the with block exits::

with TaskPool(max_workers=4) as pool:
    pool.submit(task_a, task_b, task_c)
# all tasks complete here

Note: Using the context manager does not automatically raise on task failures. Call :meth:shutdown explicitly if you need fail_on_error behaviour inside the context.

Example::

pool = TaskPool(max_workers=4)
failed = pool.submit(task_a, task_b).shutdown(fail_on_error=False)
if failed:
    ...

Initialize a :class:TaskPool.

Parameters:

Name Type Description Default
max_workers int

Maximum number of concurrent worker threads. Defaults to 10.

10

executor property

The underlying :class:~concurrent.futures.ThreadPoolExecutor.

Returns:

Type Description
ThreadPoolExecutor

The executor instance managing the thread pool.

max_workers property

Maximum number of concurrent worker threads configured at construction.

Returns:

Type Description
int

The integer value passed as max_workers to __init__.

tasks property

Read-only view of submitted tasks keyed by name.

__enter__()

Enter the context manager.

Returns:

Name Type Description
This Self

class:TaskPool instance.

__exit__(exc_type, exc_val, exc_tb)

Exit the context manager, waiting for all running tasks to finish.

Calls executor.shutdown(wait=True). Task exceptions are stored on the individual :class:Task instances but are not re-raised here — inspect :attr:Task.error or call :meth:shutdown with fail_on_error=True if you need failure propagation.

Parameters:

Name Type Description Default
exc_type type[BaseException] | None

Exception type, if any.

required
exc_val BaseException | None

Exception value, if any.

required
exc_tb TracebackType | None

Exception traceback, if any.

required

map_settled(fn, items)

Run fn(item) for every item on the pool and wait for all of them.

No :class:Task is created and :attr:tasks is unchanged. An item whose function raises an :class:Exception is logged and returned as :class:Failed; the other items still run. An item cancelled by shutdown(cancel_futures=True) is returned as :class:Failed. A BaseException from fn is not captured: it propagates from this call, and items not yet collected are left running.

Parameters:

Name Type Description Default
fn Callable[[I], T]

Called with each item on a pool thread.

required
items Iterable[I]

Inputs. Consumed fully before any item is submitted.

required

Returns:

Name Type Description
One list[Settled[T]]

class:Succeeded or :class:Failed per item, in input order.

Raises:

Type Description
AppException

If the pool has already been shut down.

Example::

for item, result in zip(paths, pool.map_settled(read, paths)):
    if isinstance(result, Failed):
        print(item, result.error)

shutdown(wait=True, cancel_futures=False, timeout=None, fail_on_error=True)

Wait for all tasks to finish and shut down the pool.

Parameters:

Name Type Description Default
wait bool

Block until all running futures complete (default True).

True
cancel_futures bool

Cancel pending (not-yet-started) futures.

False
timeout float | None

Per-task timeout in seconds when checking for exceptions.

None
fail_on_error bool

Raise :exc:~rn_forge.commons.AppException if any tasks failed.

True

Returns:

Type Description
list[TaskType]

List of failed :class:Task instances (empty if all succeeded).

Example::

failed = pool.shutdown(timeout=5, fail_on_error=False)
for task in failed:
    print(task.name, task.error)

submit(*tasks)

Submit one or more tasks for execution and return self for chaining.

Parameters:

Name Type Description Default
*tasks TaskType

:class:Task instances to enqueue.

()

Raises:

Type Description
AppException

If the pool has already been shut down.