C++20 jthread Pool: Futures, Wakeups, And Shutdown

The lock-free queue solves concurrent storage. A thread pool is bigger than that: it is a small runtime for accepting work, packaging results, waking workers, rejecting late submissions, draining accepted tasks, and shutting down cleanly.

Macro-Structure

constructor:
  create worker threads

submit():
  package user callable into a queued task
  return a future
  wake a worker

worker_loop():
  pop tasks
  run tasks
  sleep when no work exists
  exit on shutdown

destructor:
  stop accepting work
  wake workers
  join workers
  clean up retired queue nodes

The queue answers: how do producers and consumers safely push and pop tasks? The pool answers: when do workers exist, sleep, wake, accept work, reject work, and stop?

Why std::jthread

std::jthread is C++20's RAII-friendly thread type. Destroying a jthread requests cooperative stop and joins. A worker function can accept a live std::stop_token:

workers_.emplace_back([this](std::stop_token st) {
    worker_loop_(st);
});

The token is not a snapshot. It is a handle into the thread's stop state, so st.stop_requested() reflects later stop requests.

Submit Flow

submit() converts any callable plus arguments into a queued zero-argument task and returns a future to the caller.

F + Args...
  -> bind/invoke wrapper
  -> packaged_task<R()>
  -> future<R>
  -> copyable std::function<void()> queued for workers

The important pieces:

• std::invoke_result_t<F, Args...> deduces the return type.
• Forwarding references preserve lvalues and rvalues when building the task.
• std::packaged_task<R()> connects execution to a future.
• std::future<R> lets the caller wait, receive the result, or rethrow the task exception.
• A shared_ptr around the packaged task makes the wrapper copyable for std::function.

Using shared_ptr here is separate from queue-node reclamation. shared_ptr is convenient for the packaged task object. Hazard pointers are used for queue nodes because reference-count updates on every hot-path pointer copy would change the performance and progress story.

Worker Loop

An awake worker should drain available work before sleeping:

while (q_.try_dequeue(task)) {
    task();
}

The while, rather than if, matters. If a worker handled one task and immediately slept, work already in the queue might not cause another wake event.

The pool-level stop flag implements graceful shutdown: reject future submissions, finish accepted tasks, and exit once the queue is drained. An immediate-stop pool is possible, but then futures need a cancellation story.

Condition Variables

A condition variable is a sleep/wakeup mechanism, not the source of truth for work. The source of truth remains q_.try_dequeue(task).

cv_.wait(lock, predicate);

This means: while the predicate is false, unlock the mutex and sleep; after waking, relock and check again. The predicate is required because notifications do not carry meaning and spurious wakeups are allowed.

A typical worker predicate watches three things:

jthread stop requested
pool stop requested
wake_seq changed since this worker last slept

notify_one() is natural after one submit because one new task usually needs one worker. notify_all() is natural during shutdown because every worker must wake, observe stop, drain or exit, and allow the destructor to join.

Two Stop Systems

There are two related but distinct notions of stop:

A std::stop_callback can wake the condition variable if a jthread stop is requested while the worker sleeps.

Lifecycle Locks

If the queue is already thread-safe, a submit mutex is not protecting q_.enqueue(). It protects the lifecycle boundary: can this task still be accepted?

submit_mutex_:
  submit vs shutdown boundary
  "can this task be accepted?"

cv_mutex_:
  condition-variable wake boundary
  "has a wake event happened since this worker last checked?"

Without a submit/shutdown boundary, one thread could observe stop_ == false, another thread could stop and join all workers, and then the first thread could enqueue a task after workers are gone. The queue would remain structurally correct, but the returned future might never complete.

When This Pool Is The Right Story

A custom lock-free queue thread pool is not the default tool for dense numerical loops. Better defaults are OpenMP, TBB, std::execution, Kokkos, BLAS/LAPACK, cuBLAS, Thrust, or existing executor frameworks. The custom pool is most useful as a learning and systems-design project: MPMC queues, hazard pointers, futures, shutdown, condition variables, and progress under contention.