per_cpu_provider<T, Tag> and per_cpu_slot<T>
include/reloco/per_cpu_provider.hpp, include/reloco/per_cpu_slot.hpp
Per-CPU storage for kernel-style code. The C++ side of kif’s static per_cpu cells
(docs/kif-proc-language.md §4.17).
Rules
- No allocator. Per-CPU data is built without allocating and without failing.
- Zero is valid. Blocks come zero-filled; a
per_cpu_slotin zeroes is empty. - Never destroyed. A CPU does not exit, so destructors do not run.
per_cpu_slothas nodestroy(). - Atomic context.
get()and slot access areRELOCO_ATOMIC_ONLY: the caller cannot migrate (preemption or interrupts off), so the CPU does not change under the reference.reloco-tidychecks this.
per_cpu_provider<T, Tag>
| Member | Meaning |
|---|---|
get() |
The current CPU’s T &. RELOCO_ATOMIC_ONLY. Never fails. |
cpu_count() |
Number of CPUs. |
get_cpu(i) |
CPU i’s block as result<reference_wrapper<T>>; error::out_of_bounds past the end. For bring-up and reports. |
RELOCO_PER_CPU_MODEL picks the backend:
| Model | Behaviour |
|---|---|
RELOCO_PER_CPU_MODEL_SINGLE (default) |
One global block: one “CPU”. |
RELOCO_PER_CPU_MODEL_THREAD_LOCAL |
One block per thread; a thread stands in for a CPU (hosted concurrency tests). |
RELOCO_PER_CPU_MODEL_OS |
Includes detail/porting/per_cpu_provider.hpp (scaffold: per_cpu_provider.template.hpp, FreeBSD uma_zalloc_pcpu(9) + zpcpu_get(9)). |
per_cpu_slot<T>
| Member | Meaning |
|---|---|
init(f) |
Build the object with f (T(*)() noexcept). Idempotent. Call once per CPU’s block during bring-up. |
with(f) / with_mut(f) |
Run f(const T &) / f(T &). RELOCO_ATOMIC_ONLY. Using an un-inited slot trips RELOCO_ASSERT. |
is_ready() |
Whether init ran. |
Example
#include <reloco/per_cpu_provider.hpp>
#include <reloco/per_cpu_slot.hpp>
struct counters { std::uint64_t rx = 0, tx = 0; };
counters make_counters() noexcept { return counters{}; } // init_fn: cannot allocate, cannot fail
// One block per module; every per-CPU cell of the module is a field of it.
struct module_block {
reloco::per_cpu_slot<counters> stats; // zero == empty
};
struct module_tag {}; // distinguishes this module's block
using provider = reloco::per_cpu_provider<module_block, module_tag>;
void module_load() noexcept { // bring-up: every CPU's block, once
for (std::size_t cpu = 0; cpu < provider::cpu_count(); ++cpu) {
if (auto block = provider::get_cpu(cpu); block.has_value()) { // out_of_bounds cannot happen here
block->get().stats.init(&make_counters);
}
}
}
void on_rx() noexcept RELOCO_ATOMIC_ONLY { // caller did critical_enter(9): no migration
provider::get().stats.with_mut([](counters &c) noexcept { c.rx++; }); // this CPU's counters, no lock
}
Integration
- Porting. Copy
detail/porting/per_cpu_provider.template.hpptoper_cpu_provider.hpp, adapt it, and setJPLCZ_RELOCO_PORTING_HEADERS(it definesRELOCO_PER_CPU_MODEL_CUSTOM_OS). - kif.
static per_cpucells lower to onemodule_blockper module and aper_cpu_slot<T>per cell. The generated module-load code runsinitfor every CPU. A per-CPU cell may not allocate or have adropthat matters (KIFP182/186). -
Cross-CPU reads of another CPU’s data are the caller’s responsibility; use atomics for anything read remotely.
-
FreeBSD
dpcpu(9)variant.DPCPU_DEFINEneeds a literal identifier, so it cannot be stamped out fromper_cpu_provider<T, Tag>; instead each pair is opted in with macros (scaffold:detail/porting/per_cpu_provider_dpcpu.template.hpp). No allocation, but the DPCPU area is small, so keepTsmall:// net_stats.hpp: visible in every TU that uses the provider. struct net_stats { std::uint64_t rx; }; struct net_tag {}; // Tag: plain identifier, pasted into the variable name RELOCO_DECLARE_DPCPU_PER_CPU(net_stats, net_tag) // specializes per_cpu_provider + DPCPU_DECLARE // net_stats.cpp: exactly one TU (a header would give each TU its own copy). RELOCO_DEFINE_DPCPU_PER_CPU_TRIVIAL(net_stats, net_tag) // DPCPU_DEFINE: zero-filled storage // Use: atomic context only, so the CPU cannot change under the reference. critical_enter(); per_cpu_provider<net_stats, net_tag>::get().rx++; critical_exit();A
Twith a non-trivial ctor needs no special handling: the macros choose the stored type at compile time. A trivially default-constructibleTis stored as-is. Anything else is stored as raw aligned bytes, andRELOCO_DEFINE_DPCPU_PER_CPU_NONTRIVIAL(use_TRIVIALfor the first case; astatic_assertrejects the wrong one) also emits aSYSINIT(atSI_SUB_SMP) that constructsTon every CPU and aSYSUNINITthat destroys them, soget()stays a branch-free pointer computation and returnsT &either way. Using the provider before thatSYSINIThas run is invalid, and the ctor must benoexceptand non-sleeping:struct widget { widget() noexcept : ready(true) {} bool ready; void poke() noexcept; }; // non-trivial ctor struct widget_tag {}; RELOCO_DECLARE_DPCPU_PER_CPU(widget, widget_tag) // header RELOCO_DEFINE_DPCPU_PER_CPU_NONTRIVIAL(widget, widget_tag) // exactly one .cpp: storage + per-CPU init/uninit SYSINITs critical_enter(); per_cpu_provider<widget, widget_tag>::get().poke(); // already constructed on every CPU critical_exit();How it is built: all provider logic (
get(),get_cpu(),cpu_count(),init_all()/fini_all(), and thesysinit/sysuninitentry points passed toSYSINIT/SYSUNINIT) lives once in the templatedetail::per_cpu_provider_base<T, Derived>. A template cannot name aDPCPU_DEFINEd variable (it needs a literal identifier), soRELOCO_DECLARE_DPCPU_PER_CPUgenerates only aDerivedthat supplies two accessors,local()(DPCPU_PTR) andof(cpu)(DPCPU_ID_PTR). To hook the init into a differentSYSINITorder or your own module-load path, callper_cpu_provider<T, Tag>::init_all()yourself instead of using_NONTRIVIAL’sSYSINITand define the storage by hand withDPCPU_DEFINE(reloco::detail::dpcpu_storage_t<T>, reloco_dpcpu_<Tag>)(the_TRIVIALmacro’sstatic_assertwould reject a non-trivialT).
Tests
tests/test_per_cpu_slot.cpp, built once per hosted model (jplcz_reloco_per_cpu_single_tests,
jplcz_reloco_per_cpu_thread_local_tests).