Tasks
rn_forge.commons.runtime.tasks
Parallel task execution via a managed thread pool.
Provides:
Task: Abstract base class for units of concurrent work. Subclass and
implement :meth:Task.run.
TaskPool: Thread pool that submits :class:Task instances, tracks their
futures, and reports failures on shutdown. Its
:meth:TaskPool.map_settled fans a function over items.
Succeeded, Failed, Settled: Per-item outcome of :meth:TaskPool.map_settled.
Typical usage::
from rn_forge.commons.runtime.tasks import Task, TaskPool
class FetchTask(Task):
def run(self) -> None:
url = self.get_prop("url")
... # perform work
with TaskPool(max_workers=4) as pool:
pool.submit(
FetchTask("fetch-a", url="https://example.com/a"),
FetchTask("fetch-b", url="https://example.com/b"),
)
# all tasks have completed (or AppException is raised on failure)
Settled = Succeeded[T] | Failed
Outcome of one item: :class:Succeeded or :class:Failed.
Failed(error)
dataclass
Outcome of an item whose function raised an :class:Exception.
Attributes:
| Name | Type | Description |
|---|---|---|
error |
Exception
|
The exception the function raised. |
Succeeded(value)
dataclass
Outcome of an item whose function returned.
Attributes:
| Name | Type | Description |
|---|---|---|
value |
T
|
The value the function returned; |
Task(name, **props)
Bases: ABC
Abstract base for units of work executed inside a :class:TaskPool.
Subclass and implement :meth:run. Pass keyword arguments as props;
retrieve them later with :meth:get_prop.
Example::
class DownloadTask(Task):
def run(self):
url = self.get_prop("url")
...
pool = TaskPool()
pool.submit(DownloadTask("fetch-data", url="https://example.com"))
pool.shutdown()
Initialize a :class:Task.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
name
|
str
|
Unique human-readable identifier for this task. Used as the
thread name while the task is executing and as the key in
:attr: |
required |
**props
|
Any
|
Arbitrary keyword arguments stored as task properties and
retrievable via :meth: |
{}
|
error
property
The exception raised during :meth:run, if any.
Returns:
| Type | Description |
|---|---|
Exception | None
|
The caught exception, or |
Exception | None
|
not yet been run. |
future
property
The :class:~concurrent.futures.Future for this task's execution.
Returns:
| Type | Description |
|---|---|
Future[Any] | None
|
The |
Future[Any] | None
|
before the task has been submitted. |
name
property
Unique identifier for this task.
Returns:
| Type | Description |
|---|---|
str
|
The name string supplied at construction time. |
props
property
Dictionary of properties supplied at construction time.
Returns:
| Type | Description |
|---|---|
dict[str, Any]
|
A mutable dict of all keyword arguments passed to |
get_prop(key, default=None)
Return the prop value for key, or default if absent.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
key
|
str
|
The property name to look up. |
required |
default
|
Any
|
Value returned when key is not present. Defaults to
|
None
|
Returns:
| Type | Description |
|---|---|
Any
|
The stored property value, or default. |
run()
abstractmethod
Execute the task's work.
Subclasses must implement this method. It is called by the thread
pool via :meth:_start; do not call it directly. Any exception
raised here is captured in :attr:error and re-raised so the
:class:~concurrent.futures.Future records it.
TaskPool(max_workers=10)
Thread pool that submits :class:Task instances and tracks their results.
The type parameter _TaskType bounds the pool to a specific
:class:Task subclass, enabling type-safe access to task-specific
properties after completion.
Can be used as a context manager — :meth:__exit__ calls
executor.shutdown(wait=True) so all tasks finish before the with
block exits::
with TaskPool(max_workers=4) as pool:
pool.submit(task_a, task_b, task_c)
# all tasks complete here
Note:
Using the context manager does not automatically raise on task
failures. Call :meth:shutdown explicitly if you need
fail_on_error behaviour inside the context.
Example::
pool = TaskPool(max_workers=4)
failed = pool.submit(task_a, task_b).shutdown(fail_on_error=False)
if failed:
...
Initialize a :class:TaskPool.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
max_workers
|
int
|
Maximum number of concurrent worker threads.
Defaults to |
10
|
executor
property
The underlying :class:~concurrent.futures.ThreadPoolExecutor.
Returns:
| Type | Description |
|---|---|
ThreadPoolExecutor
|
The executor instance managing the thread pool. |
max_workers
property
Maximum number of concurrent worker threads configured at construction.
Returns:
| Type | Description |
|---|---|
int
|
The integer value passed as max_workers to |
tasks
property
Read-only view of submitted tasks keyed by name.
__enter__()
Enter the context manager.
Returns:
| Name | Type | Description |
|---|---|---|
This |
Self
|
class: |
__exit__(exc_type, exc_val, exc_tb)
Exit the context manager, waiting for all running tasks to finish.
Calls executor.shutdown(wait=True). Task exceptions are stored
on the individual :class:Task instances but are not re-raised
here — inspect :attr:Task.error or call :meth:shutdown with
fail_on_error=True if you need failure propagation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
exc_type
|
type[BaseException] | None
|
Exception type, if any. |
required |
exc_val
|
BaseException | None
|
Exception value, if any. |
required |
exc_tb
|
TracebackType | None
|
Exception traceback, if any. |
required |
map_settled(fn, items)
Run fn(item) for every item on the pool and wait for all of them.
No :class:Task is created and :attr:tasks is unchanged. An item
whose function raises an :class:Exception is logged and returned as
:class:Failed; the other items still run. An item cancelled by
shutdown(cancel_futures=True) is returned as :class:Failed. A
BaseException from fn is not captured: it propagates from this
call, and items not yet collected are left running.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
fn
|
Callable[[I], T]
|
Called with each item on a pool thread. |
required |
items
|
Iterable[I]
|
Inputs. Consumed fully before any item is submitted. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
One |
list[Settled[T]]
|
class: |
Raises:
| Type | Description |
|---|---|
AppException
|
If the pool has already been shut down. |
Example::
for item, result in zip(paths, pool.map_settled(read, paths)):
if isinstance(result, Failed):
print(item, result.error)
shutdown(wait=True, cancel_futures=False, timeout=None, fail_on_error=True)
Wait for all tasks to finish and shut down the pool.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
wait
|
bool
|
Block until all running futures complete (default |
True
|
cancel_futures
|
bool
|
Cancel pending (not-yet-started) futures. |
False
|
timeout
|
float | None
|
Per-task timeout in seconds when checking for exceptions. |
None
|
fail_on_error
|
bool
|
Raise :exc: |
True
|
Returns:
| Type | Description |
|---|---|
list[TaskType]
|
List of failed :class: |
Example::
failed = pool.shutdown(timeout=5, fail_on_error=False)
for task in failed:
print(task.name, task.error)
submit(*tasks)
Submit one or more tasks for execution and return self for chaining.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
*tasks
|
TaskType
|
:class: |
()
|
Raises:
| Type | Description |
|---|---|
AppException
|
If the pool has already been shut down. |