Multispecies cache management — a Goldman-equation policy as a PluginCache, and two notes on byte-miss-ratio

[flatmax]

Hi — I've been using libCacheSim (via the Python bindings) as the evaluation harness for a cache-replacement policy derived from electrodiffusion across a biological membrane, and wanted to share both the artefact and a couple of findings about the simulator's metrics that others here might find useful.

Paper: arXiv:2608.14561 — A Biophysically-Inspired Feedback Controller for Multi-Class Cache Fairness
Code: https://github.com/flatmax/membrane.cache

Multispecies, not just multi-class

The paper is titled in the systems vocabulary, but the mechanism is better described as multispecies cache management, and the distinction is what makes it more than a metaphor.

Treat the boundary between a cache's active and evictable regions as a semi-permeable membrane. Content classes — system prompts, user documents, code context, conversation history in an LLM-serving cache — are ionic species crossing it. Tokens are the universal mass every object carries regardless of species, so they sum into a single membrane potential V.

That single V is the whole point. In multispecies electrodiffusion there is one membrane and therefore exactly one potential; you don't get to give each species its own. Species are coupled through it and compete through it. Each species k has its own concentrations and its own reversal potential V_rev_k = V_T · ln(c_l_k / c_u_k), and its flux under the shared V is

Φ_k = P_k · V · (c_l_k − c_u_k · exp(−V/V_T)) / (1 − exp(−V/V_T))

The membrane equilibrates when the summed flux is zero — that's the Goldman equation determining equilibrium V_m — at which point individual species generally aren't at zero flux; they're flowing in directions that balance out. An underrepresented species sees a strong upward driving force, a balanced one sees almost none.

So the fairness behaviour is an emergent equilibrium property rather than something imposed. There are no per-class quotas, weights, or thresholds — in the headline configuration the permeability P_k = P is pinned identical across all species, and the per-species reversal potentials do all the work. This is also why it isn't a partitioning scheme: UCP-style partitions isolate species from each other, whereas a shared V makes them negotiate.

Worth saying plainly that this isn't reverse-engineered. The multispecies generalisation is classical biophysics (Goldman 1943; Hodgkin & Katz 1949). The single-species GHK form is the special case, and it's what earlier tracks of this work used before there were multiple classes to model.

The headline number: on synthetic multispecies workloads across four skew levels, it closes 27–72% of the LRU→Belady gap on the worst-served-species miss ratio.

How it's wired into libCacheSim

A standard PluginCache with the six hooks. Two variants live in libcachesim_test/flux_plugin.py:

• flux — spec-faithful. Maintains a persistent ACTIVE shadow, evaluates Φ once per turn boundary (detected via clock_time changes in the hit/miss hooks), and drains the flux accumulator in an internal loop.
• flux_lite — a simpler variant that re-evaluates Φ on every eviction call and lets libCacheSim's outer budget loop do the draining.

Everything runs against stock baselines (LRU, ARC, S3-FIFO, SIEVE, FIFO, Belady) on the same trace with the same scalar cache_size, and the whole thing is reproducible from seeds. The trace reader is a ReaderProtocol adapter over our own workload generator; next_access_vtime is populated so Belady is a real oracle.

Two observations that might interest this project

These come from the single-species reduction — one membrane, one species, which is exactly the flat cache libCacheSim models.

1. byte-miss-ratio can be blind to policies that disagree on 100% of evictions. I added shadow-LRU telemetry to the plugin — at every eviction call, compute what plain LRU would have evicted from the current residency set and compare. Result on held-out seeds: the spec-faithful variant picks a different victim from LRU on 99.6% of eviction-path calls, yet lands at byte_mr 0.1423 vs LRU's 0.1424. flux_lite disagrees on 17.5% and also ties.

I swept cache fraction over [0.125, 0.25, 0.5, 0.75] to check whether byte_mr ever separates them. It doesn't, at any fraction. At 0.75 every non-degenerate policy hits Belady's floor; at 0.125 the cluster widens but flux stays inside it. Meanwhile the object miss ratio — collected all along by process_trace, and which I'd simply never looked at — shows the same policy closing ~53% of the LRU→Belady gap at frac=0.25 and ~31% at frac=0.125 (8 test seeds).

The mechanism is legible: the flux equation weighs tokens as the "voltage", and the mover-pick tiebreak order propagates that into a bias toward evicting larger residents first, so you get fewer distinct misses at comparable byte volume. My takeaway is methodological rather than a criticism of the simulator — when a policy's structure is size-sensitive, byte_mr and obj_mr can rank it very differently, and reporting only the former can make an active mechanism look inert. Both are right there in the output.

2. A 3-seed average moved S3-FIFO from best to worst. At 3 test seeds on my workload, S3-FIFO was the best non-oracle policy (0.1694). At 8 seeds it was the worst (0.1657) with roughly double the stdev of every other policy (±0.038 vs ±0.02). The 3-seed edge was seed-specific. This says nothing bad about S3-FIFO — my synthetic workload just happens to produce high-variance results for its small/main/ghost structure at these sizes — but it made me stop trusting low-seed-count averages, and it might be worth a note in the docs for others doing quick comparisons.

Two questions

• Per-species metrics. The multispecies result needs per-species miss ratios, which I currently compute outside libCacheSim by post-processing. There's no notion of a request carrying a species label in the core as far as I can tell. Is there any appetite for that, or an existing pattern I've missed? Happy to contribute if there's interest in the shape.
• Turn-boundary semantics. The spec-faithful plugin depends on detecting turn boundaries via clock_time transitions inside the hit/miss hooks. Is that a stable thing to rely on, or is it incidental to the current reader implementations?

Thanks for libCacheSim — having validated reference implementations of the baselines is what made the honest null result in (1) trustworthy enough to publish rather than something I'd have quietly assumed was my own bug.