Memory Management

What Each Pool Holds

The two pools store different halves of an entry. The key pool holds the keys: one slot per cached key, carrying the key itself (up to 48 bytes) together with its hash, TTL, hit count and last-access time; the value is only referenced through a pointer to the value pool. The value pool holds the values: every stored value occupies one block of serialized bytes — the compressed form when the entry went through yac.compress_threshold.

The one exception is embedded values: tiny scalars — null, true, false, most integers (those that fit in 60 signed bits on 64-bit builds), strings of up to 7 bytes and empty arrays — are stored directly inside the slot, in the pointer that would otherwise reference a block. Such values occupy no space in the value pool at all; only their key does.

As a rough sizing guide:

• the key pool holds around 8,000 keys per MB, so size yac.keys_memory_size as the number of distinct keys divided by 8,000 per MB, rounded up — the default 8M holds around 64,000 keys;

• the value pool must hold every value that may still be read, so size yac.values_memory_size as the number of live values times their average serialized size (after compression), and allow roughly twice that: the pool is a ring, and a value only dies once the allocator cursor comes back around to overwrite it.

Embedded values occupy a slot like any other entry, but consume no space in the value pool, so leave them out of the second calculation.

The key pool (slots)

yac.keys_memory_size holds a fixed-size table of slots — the default of 8M gives around 65,536 slots. Each stored key occupies exactly one slot, so this pool caps the number of entries that can exist at once; unlike the value pool, slots are never individually freed. An expired slot — one past its TTL, or the tombstone left by Yac::delete() — is recycled for free when a new key needs it. Only when all four candidate slots of a probe path hold live entries is one of them evicted to make room — one kick (the kicks counter of Yac::info()).

The eviction picks among the four live candidates of the colliding probe path only:

• the least recently used one (the oldest atime) is evicted;

• on a tie the least-hit entry, then the earliest probe position.

A common point of confusion: slots_used reaching slots_size is not an error condition. A cache whose working set of keys is larger than the slot table simply runs at 100% occupancy from then on, evicting and re-inserting as needed. The only thing that says whether the key pool is sized correctly is the hit rate (hits / (hits + miss), computed over the deltas between two Yac::info() snapshots rather than the lifetime average). A high kicks count on its own means nothing is wrong — the key distribution is simply not uniform and some probe paths collide more than others. Only when the hit rate and kicks are both bad is the table too small for the key set, and the remedy is a bigger yac.keys_memory_size.

A second consequence of slots never being freed: entries with no TTL (ttl = 0) that are never read again keep occupying a slot until an eviction happens to pick them. If an application stores large amounts of such one-shot data, give those entries a TTL so they expire and can be recycled without displacing live entries, or size the key pool for the full key set.

The value pool (segments)

yac.values_memory_size is split into segments of 4M each, managed as rings: writes advance a per-segment cursor and space is never freed per entry. When an allocation no longer fits, the cursor wraps back to the start of a segment — one recycle (the recycles counter of Yac::info()). A recycle does not invalidate the segment at once: overwritten values stay readable until the wrapped cursor actually overwrites them, at which point their reads fail the integrity guard and turn into misses.

Two sizes matter for this pool: the total yac.values_memory_size must hold the working set of live values, and a single entry can hold at most 1 MB as stored (YAC_MAX_RAW_COMPRESSED_LEN). Values larger than that are therefore always compressed before being stored; a value that cannot shrink below 1 MB — most often because it is random data — is rejected and bumps the fails counter. The absolute size limit on the value itself is much higher: serialized values above 64 MB (YAC_MAX_VALUE_RAW_LEN, that is (1 << 26) - 1 bytes) are rejected outright.

Sizing and what to watch

Start with the defaults and watch the counters of Yac::info() — they accumulate from start_time, so compare two snapshots taken some time apart:

• hit rate healthy (say >= 90%): the cache is fine; nothing to do, whatever the other counters show;

• hit rate low and kicks climbing: the key pool is too small for the key set — live entries get evicted before they are re-read. Raise yac.keys_memory_size;

• recycles frequent: this is a real problem, not a benign counter. A recycle means the value allocator has wrapped and is about to overwrite entries — anything overwritten dies before it could be re-read, so the bytes spent storing it were wasted and the hit rate suffers. The value pool is too small for the volume of live data. In order of impact:

  • give entries a TTL. Values written with ttl = 0 stay live forever, so they keep occupying the pool and force the cursor to wrap sooner. A TTL bounds how long each entry may live, shrinking the live working set the pool has to hold;

  • raise yac.values_memory_size so the pool holds the whole live value set (remember to budget roughly twice the live footprint — a value only dies once the cursor comes back around to overwrite it);

  • store less per entry: lower yac.compress_threshold if it is set above the 1024 minimum, so large payloads are compressed, and trim values that do not need to be cached in full;

• fails growing: values that could not be stored, most often a single value larger than the 1 MB stored-size limit even after compression — split the value.